Connectionist models of development

Finding new facts; thinking new thoughts

The idea of the child as an active learner is one of Piaget's enduring legacies. In this chapter, I discuss the ways in which contemporary computational models of learning do, and do not, address learning as an active, child-driven process. In Part 1, I discuss the problem of search and exploration. In Part 2, I discuss the (harder and more interesting) problem of hypothesis generation. I conclude by proposing some possible new directions for research.

Perseveration and the status of 3-year-olds' knowledge in a card-sorting task: evidence from studies involving congruent flankers

Infants and young children often perseverate despite apparent knowledge of the correct response. Two Experiments addressed questions concerning the status of such knowledge in the context of a card-sorting task. In Experiment 1, three groups of 3-year-olds sorted bivalent cards one way and then were instructed to switch and sort the same cards using new rules under varying conditions of support offered by congruent flankers. Although formal aspects of the task such as higher-order rule use, stimulus redescription, and dimensional shifting remained constant across all conditions, use of the new rules increased with parametric increases in environmental support for the use of the new rules. In Experiment 2, 3-year-olds were more likely to switch and use new rules when test stimuli were flanked by congruent flankers rather than neutral flankers, even though both conditions made equivalent demands on attentional inhibition. Thus, in both experiments, children's knowledge of the new rules proved to be adequate under less demanding conditions but inadequate under more demanding conditions. These findings are consistent with the idea that children's knowledge is graded in strength rather than present or absent.

Facing changes and changing faces in adolescence: a new model for investigating adolescent-specific interactions between pubertal, brain and behavioral development

Adolescence is a time of dramatic physical, cognitive, emotional, and social changes as well as a time for the development of many social-emotional problems. These characteristics raise compelling questions about accompanying neural changes that are unique to this period of development. Here, we propose that studying adolescent-specific changes in face processing and its underlying neural circuitry provides an ideal model for addressing these questions. We also use this model to formulate new hypotheses. Specifically, pubertal hormones are likely to increase motivation to master new peer-oriented developmental tasks, which will in turn, instigate the emergence of new social/affective components of face processing. We also predict that pubertal hormones have a fundamental impact on the re-organization of neural circuitry supporting face processing and propose, in particular, that, the functional connectivity, or temporal synchrony, between regions of the face-processing network will change with the emergence of these new components of face processing in adolescence. Finally, we show how this approach will help reveal why adolescence may be a period of vulnerability in brain development and suggest how it could lead to prevention and intervention strategies that facilitate more adaptive functional interactions between regions within the broader social information processing network.

Twenty years and going strong: A dynamic systems revolution in motor and cognitive development

This article reviews the major contributions of dynamic systems theory in advancing thinking about development, the empirical insights the theory has generated, and the key challenges for the theory on the horizon. The first section discusses the emergence of dynamic systems theory in developmental science, the core concepts of the theory, and the resonance it has with other approaches that adopt a systems metatheory. The second section reviews the work of Esther Thelen and colleagues, who revolutionized how researchers think about the field of motor development. It also reviews recent extensions of this work to the domain of cognitive development. Here, the focus is on dynamic field theory, a formal, neurally grounded approach that has yielded novel insights into the embodied nature of cognition. The final section proposes that the key challenge on the horizon is to formally specify how interactions among multiple levels of analysis interact across multiple time scales to create developmental change.

Infant joint attention, neural networks and social cognition

Neural network models of attention can provide a unifying approach to the study of human cognitive and emotional development (Posner & Rothbart, 2007). In this paper we argue that a neural network approach to the infant development of joint attention can inform our understanding of the nature of human social learning, symbolic thought process and social cognition. At its most basic, joint attention involves the capacity to coordinate one's own visual attention with that of another person. We propose that joint attention development involves increments in the capacity to engage in simultaneous or parallel processing of information about one's own attention and the attention of other people. Infant practice with joint attention is both a consequence and an organizer of the development of a distributed and integrated brain network involving frontal and parietal cortical systems. This executive distributed network first serves to regulate the capacity of infants to respond to and direct the overt behavior of other people in order to share experience with others through the social coordination of visual attention. In this paper we describe this parallel and distributed neural network model of joint attention development and discuss two hypotheses that stem from this model. One is that activation of this distributed network during coordinated attention enhances the depth of information processing and encoding beginning in the first year of life. We also propose that with development, joint attention becomes internalized as the capacity to socially coordinate mental attention to internal representations. As this occurs the executive joint attention network makes vital contributions to the development of human symbolic thinking and social cognition.

Using cluster analysis to interpret the variability of gross motor scores of children with typical development

BACKGROUND

Longitudinal research on gross motor percentile rank scores of children with typical development has documented intra-individual variability of scoring patterns. Clinically, interpreting these fluctuations presents a challenge for therapists.

OBJECTIVE

The aim of this study was to determine the utility of cluster analysis as a technique to organize the gross motor scoring patterns of children with typical development into clinically relevant groups.

DESIGN

This was a descriptive, exploratory study using data from 2 longitudinal studies.

PARTICIPANTS

Sixty-six children with typical development participated in the study.

METHODS

The children were assessed on the gross motor subscale of the Peabody Developmental Motor Scales at 9, 11, 13, 16, and 21 months of age and on the gross motor subscale of the Peabody Developmental Motor Scales, 2nd edition, at 4, 4.5, 5, and 5.5 years of age. Demographic and health data were collected. Parents were interviewed when the children were 8 years of age. Cluster analysis was conducted. Demographic and health data were compared across clusters.

RESULTS

Four distinct and clinically relevant clusters were identified. A significant difference was found among the clusters for total number of illnesses.

LIMITATIONS

The children in these analyses were at low risk for gross motor problems. Further research with a more high-risk sample is needed to validate the clinical utility of the identified clusters.

CONCLUSIONS

Cluster analysis techniques may offer a mechanism to explore longitudinal data in physical therapy research. The techniques provided a mechanism to group data without losing the richness of information provided by the intra-individual variability of scoring patterns. Clinically, examination of distinct scoring patterns may lead to improved accuracy in screening for gross motor concerns compared with the traditional use of single-assessment cutoff points.

Computational perspectives on cognitive development

This article reviews the efforts to develop process models of infants' and children's cognition. Computational process models provide a tool for elucidating the causal mechanisms involved in learning and development. The history of computational modeling in developmental psychology broadly follows the same trends that have run throughout cognitive science-including rule-based models, neural network (connectionist) models, ACT-R models, ART models, decision tree models, reinforcement learning models, and hybrid models among others. Copyright © 2010 John Wiley & Sons, Ltd. For further resources related to this article, please visit the WIREs website.

Interpreting developmental changes in neuroimaging signals

The imaging of developmental changes in brain function is challenging, but great strides have been made in addressing many of the conceptual issues that this work raises. I highlight a set of issues that remain to be addressed in this literature. First, I argue that the appeal to developmental neurobiology is often misplaced, as it focuses on neurodevelopmental processes that are mostly completed by the age at which neuroimaging studies can be performed. Second, I argue that the concept of "normative" development needs to be reexamined, as it reflects fundamental value judgments about brain development that seem inappropriate for scientific investigation. Third, I examine the ways in which developmental changes are often interpreted, arguing that common interpretations, including the concepts of "efficiency" and "focalization" may be less useful than commonly supposed. To put developmental neuroimaging on stronger footing, we need to develop stronger connections between computational and neurobiological accounts of developmental changes.

Understanding genetic, neurophysiological, and experiential influences on the development of executive functioning: the need for developmental models

Flexibility is a cornerstone of adaptive behavior and is made possible by a family of processes referred to collectively as executive functions. Executive functions vary in efficacy from individual to individual and also across developmental time. Infants and young children, for example, have difficulty flexibly adapting their behavior, and often repeat actions that are no longer appropriate. And although older children do not typically make such striking errors, they have more difficulty exercising control than adolescents and adults. Such developmental variability parallels (at least in some respects) inter-individual variability in executive functions. Individuals who suffer damage or dysfunction in regions of the prefrontal cortex, for example, often experience difficulty in flexibly adapting their behavior to changes in context. As well, genetic differences between individuals are strongly associated with differences in executive control. Parallels between developmental and inter-individual variability suggest hypotheses about possible mechanisms underlying the development of executive functions but carry risks when interpreted improperly. Overcoming these pitfalls will require mechanistic characterizations of executive functioning that are more deeply rooted in developmental principles. Copyright © 2010 John Wiley & Sons, Ltd. For further resources related to this article, please visit the WIREs website.

Solving the paradox of the equipotential and modular brain: a neurocomputational model of stroke vs. slow-growing glioma

In acute brain damage (e.g., stroke), patients can be left with specific deficits while other domains are unaffected, consistent with the classical 'modular' view of cortical organization. On this view, relearning of impaired function is limited because the remaining brain regions, tuned to other domains, have minimal capacity to assimilate an alternative activity. A clear paradox arises in low-grade glioma where an even greater amount of cortex may be affected and resected without impairment. Using a neurocomputational model we account for the modular nature of normal function as well as the contrasting types of brain insult through the interaction of three computational principles: patterns of connectivity; experience-dependent plasticity; and the time course of damage. This work provides support for a neo-Lashleyan view of cortical organization.

The growth of tense productivity

PURPOSE

This study tests empirical predictions of a maturational model for the growth of tense in children younger than 36 months using a type-based productivity measure.

METHOD

Caregiver-child language samples were collected from 20 typically developing children every 3 months from 21 to 33 months of age. Growth in the productivity of tense morphemes, centered at 21 months, was examined using hierarchical linear modeling. The empirical Bayes residuals from 21- to 30-month productivity growth trajectories predicted children's accuracy of tense marking at 33 months.

RESULTS

A random effects quadratic growth model with no intercept best characterized the growth of tense marking between 21 and 30 months. Average development was characterized by slow instantaneous linear growth of less than 1 morpheme per month at 21 months and acceleration overall. Significant variation around this trend was also evident. Children's linear and quadratic empirical Bayes residuals together predicted 33-month accuracy scores (r = .672, p = .008).

CONCLUSIONS

Acceleration and variation about this trend are consistent with maturational models of language acquisition. With an empirically sound characterization of early variation in morphosyntactic growth rates, future investigations can more rigorously test hypotheses regarding biological, environmental, and developmental contributions to the acquisition of morphosyntax.

A parallel and distributed-processing model of joint attention, social cognition and autism

The impaired development of joint attention is a cardinal feature of autism. Therefore, understanding the nature of joint attention is central to research on this disorder. Joint attention may be best defined in terms of an information-processing system that begins to develop by 4-6 months of age. This system integrates the parallel processing of internal information about one's own visual attention with external information about the visual attention of other people. This type of joint encoding of information about self and other attention requires the activation of a distributed anterior and posterior cortical attention network. Genetic regulation, in conjunction with self-organizing behavioral activity, guides the development of functional connectivity in this network. With practice in infancy the joint processing of self-other attention becomes automatically engaged as an executive function. It can be argued that this executive joint attention is fundamental to human learning as well as the development of symbolic thought, social cognition and social competence throughout the life span. One advantage of this parallel and distributed-processing model of joint attention is that it directly connects theory on social pathology to a range of phenomena in autism associated with neural connectivity, constructivist and connectionist models of cognitive development, early intervention, activity-dependent gene expression and atypical ocular motor control.

The dynamic nature of knowledge: insights from a dynamic field model of children's novel noun generalization

This paper examines the tie between knowledge and behavior in a noun generalization context. An experiment directly comparing noun generalizations of children at the same point in development in forced-choice and yes/no tasks reveals task-specific differences in the way children's knowledge of nominal categories is brought to bear in a moment. To understand the cognitive system that produced these differences, the real-time decision processes in these tasks were instantiated in a dynamic field model. The model captures both qualitative and quantitative differences in performance across tasks and reveals constraints on the nature of children's accumulated knowledge. Additional simulations of developmental change in the yes/no task between 2 and 4 years of age illustrate how changes in children's representations translate into developmental changes in behavior. Together, the empirical data and model demonstrate the dynamic nature of knowledge and are consistent with the perspective that knowledge cannot be separated from the task-specific processes that create behavior in the moment.

Précis ofSemantic Cognition: A Parallel Distributed Processing Approach

In this précis of our recent book,Semantic Cognition: A Parallel Distributed Processing Approach(Rogers & McClelland 2004), we present a parallel distributed processing theory of the acquisition, representation, and use of human semantic knowledge. The theory proposes that semantic abilities arise from the flow of activation among simple, neuron-like processing units, as governed by the strengths of interconnecting weights; and that acquisition of new semantic information involves the gradual adjustment of weights in the system in response to experience. These simple ideas explain a wide range of empirical phenomena from studies of categorization, lexical acquisition, and disordered semantic cognition. In this précis we focus on phenomena central to the reaction against similarity-based theories that arose in the 1980s and that subsequently motivated the “theory-theory” approach to semantic knowledge. Specifically, we consider (1) how concepts differentiate in early development, (2) why some groupings of items seem to form “good” or coherent categories while others do not, (3) why different properties seem central or important to different concepts, (4) why children and adults sometimes attest to beliefs that seem to contradict their direct experience, (5) how concepts reorganize between the ages of 4 and 10, and (6) the relationship between causal knowledge and semantic knowledge. The explanations our theory offers for these phenomena are illustrated with reference to a simple feed-forward connectionist model. The relationships between this simple model, the broader theory, and more general issues in cognitive science are discussed.

Long-term trajectories of the development of speech sound production in pediatric cochlear implant recipients

PURPOSE

This study characterized the development of speech sound production in prelingually deaf children with a minimum of 8 years of cochlear implant (CI) experience.

METHOD

Twenty-seven pediatric CI recipients' spontaneous speech samples from annual evaluation sessions were phonemically transcribed. Accuracy for these speech samples was evaluated in piecewise regression models.

RESULTS

As a group, pediatric CI recipients showed steady improvement in speech sound production following implantation, but the improvement rate declined after 6 years of device experience. Piecewise regression models indicated that the slope estimating the participants' improvement rate was statistically greater than 0 during the first 6 years postimplantation, but not after 6 years. The group of pediatric CI recipients' accuracy of speech sound production after 4 years of device experience reasonably predicts their speech sound production after 5-10 years of device experience.

CONCLUSIONS

The development of speech sound production in prelingually deaf children stabilizes after 6 years of device experience, and typically approaches a plateau by 8 years of device use. Early growth in speech before 4 years of device experience did not predict later rates of growth or levels of achievement. However, good predictions could be made after 4 years of device use.

Generalizing the dynamic field theory of spatial cognition across real and developmental time scales

Within cognitive neuroscience, computational models are designed to provide insights into the organization of behavior while adhering to neural principles. These models should provide sufficient specificity to generate novel predictions while maintaining the generality needed to capture behavior across tasks and/or time scales. This paper presents one such model, the dynamic field theory (DFT) of spatial cognition, showing new simulations that provide a demonstration proof that the theory generalizes across developmental changes in performance in four tasks-the Piagetian A-not-B task, a sandbox version of the A-not-B task, a canonical spatial recall task, and a position discrimination task. Model simulations demonstrate that the DFT can accomplish both specificity-generating novel, testable predictions-and generality-spanning multiple tasks across development with a relatively simple developmental hypothesis. Critically, the DFT achieves generality across tasks and time scales with no modification to its basic structure and with a strong commitment to neural principles. The only change necessary to capture development in the model was an increase in the precision of the tuning of receptive fields as well as an increase in the precision of local excitatory interactions among neurons in the model. These small quantitative changes were sufficient to move the model through a set of quantitative and qualitative behavioral changes that span the age range from 8 months to 6 years and into adulthood. We conclude by considering how the DFT is positioned in the literature, the challenges on the horizon for our framework, and how a dynamic field approach can yield new insights into development from a computational cognitive neuroscience perspective.

Re-thinking stages of cognitive development: An appraisal of connectionist models of the balance scale task

The present paper re-appraises connectionist attempts to explain how human cognitive development appears to progress through a series of sequential stages. Models of performance on the Piagetian balance scale task are the focus of attention. Limitations of these models are discussed and replications and extensions to the work are provided via the Cascade-Correlation algorithm. An application of multi-group latent class analysis for examining performance of the networks is described and these results reveal fundamental functional characteristics of the networks. Evidence is provided that strongly suggests that the networks are unable to acquire a mastery of torque and, although they do recover certain rules of operation that humans do, they also show a propensity to acquire rules never previously seen.

Grammar and the Lexicon: Developmental Ordering in Language Acquisition

Recent accounts of language acquisition propose that the knowledge structures that comprise language develop within a single, unified system that shares computational resources and representations. One implication of this approach is that developmental relations within the system become central to theorizing about language acquisition. Previous work suggested that lexical development preceded grammatical development, a developmental ordering with strong theoretical implications. One purpose of the current article is to test this developmental ordering hypothesis. Results showed that children (aged 16-30 months) developed lexicon and grammar synchronously. The second purpose is to demonstrate a recently developed method for testing developmental ordering, the nonlinear-mapping approach, and show how the method can be extended to capitalize on multiply determined developmental systems, such as language.

Modeling developmental cognitive neuroscience

In the past few years connectionist models have greatly contributed to formulating theories of cognitive development. Some of these models follow the approach of developmental cognitive neuroscience in exploring interactions between brain development and cognitive development by integrating structural change into learning. We describe two classes of these models. The first focuses on experience-dependent structural elaboration within a brain region by adding or deleting units and connections during learning. The second models the gradual integration of different brain areas based on combinations of experience-dependent and maturational factors. These models provide new theories of the mechanisms of cognitive change in various domains and they offer an integrated framework to study normal and abnormal development, and normal and impaired adult processing.

Approximate Optimal Control as a Model for Motor Learning

Current models of psychological development rely heavily on connectionist models that use supervised learning. These models adapt network weights when the network output does not match the target outputs computed by some agent. The authors present a model of motor learning in which the child uses exploration to discover appropriate ways of responding. The model is consistent with what is known about how neural systems evaluate behavior. The authors model the development of reaching and investigate N. Bernstein's (1967) hypotheses about early motor learning. Simulations show the course of learning as well as model the kinematics of reaching by a dynamical arm.

Hebbian learning and development

Hebbian learning is a biologically plausible and ecologically valid learning mechanism. In Hebbian learning, 'units that fire together, wire together'. Such learning may occur at the neural level in terms of long-term potentiation (LTP) and long-term depression (LTD). Many features of Hebbian learning are relevant to developmental theorizing, including its self-organizing nature and its ability to extract statistical regularities from the environment. Hebbian learning mechanisms may also play an important role in critical periods during development, and in a number of other developmental phenomena.

Testing the dynamic field theory: working memory for locations becomes more spatially precise over development

The dynamic field theory predicts that biases toward remembered locations depend on the separation between targets, and the spatial precision of interactions in working memory that become enhanced over development. This was tested by varying the separation between A and B locations in a sandbox. Children searched for an object 6 times at an A location, followed by 3 trials at a B location. Two- and 4-year-olds', but not 6-year-olds', responses were biased toward A when A and B were 9-in. and 6-in. apart. When A and B were separated by 2 in., however, 4- and 6-year-olds' responses were biased toward A. Thus, the separation at which responses were biased toward A decreased across age groups, supporting the predictions of the theory.

Developmental continuity in the processes that underlie spatial recall

This study investigated whether children's spatial recall performance shows three separable characteristics: (1) biases away from symmetry axes (geometric effects); (2) systematic drift over delays; and (3) biases toward the exemplar distribution experienced in the task (experience-dependent effects). In Experiment 1, the location of one target within each geometric category was varied. Children's responses showed biases away from a midline axis that increased over delays. In Experiment 2, multiple targets were placed within each category at the same locations used in Experiment 1. After removing geometric effects, 6-year-olds'--but not 11-year-olds'--responses were biased toward the average remembered location over learning. In Experiment 3, children responded to one target more frequently than the others. Both 6- and 11-year-olds showed biases toward the most frequent target over learning. These results provide a bridge between the performance of younger children and adults, demonstrating continuity in the processes that underlie spatial memory abilities across development.