The serial-parallel dilemma: a case study in a linkage of theory and method

Quantifying Entropy in Response Times (RT) Distributions Using the Cumulative Residual Entropy (CRE) Function

Response times (RT) distributions are routinely used by psychologists and neuroscientists in the assessment and modeling of human behavior and cognition. The statistical properties of RT distributions are valuable in uncovering unobservable psychological mechanisms. A potentially important statistical aspect of RT distributions is their entropy. However, to date, no valid measure of entropy on RT distributions has been developed, mainly because available extensions of discrete entropy measures to continuous distributions were fraught with problems and inconsistencies. The present work takes advantage of the cumulative residual entropy (CRE) function-a well-known differential entropy measure that can circumvent those problems. Applications of the CRE to RT distributions are presented along with concrete examples and simulations. In addition, a novel measure of instantaneous CRE is developed that captures the rate of entropy reduction (or information gain) from a stimulus as a function of processing time. Taken together, the new measures of entropy in RT distributions proposed here allow for stronger statistical inferences, as well as motivated theoretical interpretations of psychological constructs such as mental effort and processing efficiency.

Merging the Psychophysical Function With Response Times for Auditory Detection of One vs. Two Tones

The purpose of this study is to take preliminary steps to unify psychoacoustic techniques with reaction-time methodologies to address the perceptual mechanisms responsible for the detection of one vs. multiple sounds. We measured auditory redundancy gains for auditory detection of pure tones widely spaced in frequency using the tools of Systems Factorial Technology to evince the system architecture and workload capacity in two different scenarios (SOFT and LOUD). We adopted an experimental design in which the presence or absence of a target at each of two frequencies was combined factorially with two stimulus levels. Replicating previous work, results did not allow an assessment of system architecture due to a failure to observe factor influence at the level of distribution ordering for dual-target stimuli for both SOFT and LOUD scenarios. All subjects demonstrated very modest redundancy gains for the dual-target compared to the single-target stimuli, and results were similar for both LOUD and SOFT. We propose that these results can be predicted by a mental architecture that falls into the class of integrated subadditive parallel systems, using a well-supported assumption that reaction time is driven by loudness. We demonstrate that modeled loudness of the experimental sounds (which ranged between about 0.2 and 14 sones) is highly correlated with mean reaction time (r = −0.87), and we provide a proof-of-concept model based on Steven’s Power law that predicts both a failure of distributional ordering for dual-target stimuli and very modest redundancy gains.

The impact of simulated hemianopia on visual search for faces, words, and cars

Tests of visual search can index the effects of perceptual load and compare the processing efficiency for different object types, particularly when one examines the set-size effect, the increase in search time for each additional stimulus in an array. Previous studies have shown that the set-size effect is increased by manoeuvres that impede object processing, and in patients with object processing impairments. In this study, we examine how the low-level visual impairment of hemianopia affects visual search for complex objects, using a virtual paradigm. Forty-two healthy subjects performed visual search for faces, words, or cars with full-viewing as well as gaze-contingent simulations of complete left or right hemianopia. Simulated hemianopia lowered accuracy and discriminative power and increased response times and set-size effects, similarly for faces, words and cars. A comparison of set-size effects between target absent and target present trials did not show a difference between full-view and simulated hemianopic conditions, and a model of decision-making suggested that simulated hemianopia reduced the rate of accumulation of perceptual data, but did not change decision thresholds. We conclude that simulated hemianopia reduces the efficiency of visual search for complex objects, and that such impairment should be considered when interpreting results from high-level object processing deficits.

When two faces are not better than one: Serial limited-capacity processing with redundant-target faces

Many researchers believe that faces-whether presented alone or as part of an ensemble-are processed automatically. According to this idea, (a) the detection of single or multiple faces is resource free and does not require allocation of attention, and (b) visual search for faces is held in parallel. The current study put these hypotheses under direct scrutiny. Participants performed in a redundant target-detection task, responding according to the presence or absence of a face (or faces) on the display. I used a rigorous methodology known as the system factorial technology (SFT). The SFT methodology afforded simultaneous assessment of (a) architecture (serial vs. parallel), (b) stopping rule (exhaustive vs. self-terminating), and (c) capacity (limited, unlimited, or supercapacity). SFT analyses were held at the level of the mean RTs and at the level of the RT distributions. The analyses pointed conclusively to a serial self-terminating architecture with limited capacity. These findings cast serious doubts on the alleged automaticity of face perception.

The cognitive architecture of processes responsible to assess similarity and clarity in a comparison task

When asked to compare two stimuli, participants are on average faster to respond Same than Different, an effect coined the fast-same. The dual-process theory argues that information about similarity is processed in priority over any other type of information, causing the fast-same effect. We tested this serial architecture of cognitive processes using a double factorial paradigm, suitable for a Systems Factorial Technology (SFT) analysis. Twenty participants completed a task in which they compared two letters, which were varied on two dimensions: the similarity and the clarity of the letters. Their task was to indicate if the second letter was the Same as the second letter (ranging from identical and clear to similar and slightly blurry) or if it was Different (if the stimuli were either dissimilar or very blurry). The SFT results show that most participants processed the information in serial, but in a mixed order. In other words, for some trials, participants processed similarity first, and for some other trials, they processed clarity first. This implies that participant indeed processed information in serial in the comparison task, but that it does not cause the fast-same effect.

What do cows drink? A systems factorial technology account of processing architecture in memory intersection problems

It has long been known that cues can be used to improve performance on memory recall tasks. There is evidence to suggest additional cues provide further benefit, presumably by narrowing the search space. Problems that require integration of two or more cues, alternately referred to as memory intersections or multiply constrained memory problems, could be approached using several strategies, namely serial or parallel consideration of cues. The type of strategy implicated is essential information for the development of theories of memory, yet evidence to date has been inconclusive. Using a novel application of the powerful Systems Factorial Technology (Townsend & Nozawa, 1995) we find strong evidence that participants use two cues in parallel in free recall tasks - a finding that contradicts two recent publications in this area. We then provide evidence from a related recognition task showing that while most participants also use a parallel strategy in that paradigm, a reliable subset of participants used a serial strategy. Our findings suggest a theoretically meaningful distinction between participants strategies in recall and recognition based intersection memory tasks, and also highlight the importance of tightly controlled methodological and analytic frameworks to overcome issues of serial/parallel model mimicry.

A Limiting Channel Capacity of Visual Perception: Spreading Attention Divides the Rates of Perceptual Processes

This study investigated effects of divided attention on the temporal processes of perception. During continuous watch periods, observers responded to sudden changes in the color or direction of any one of a set of moving objects. The set size of moving objects was a primary variable. A simple detection task required responses to any display change, and a selective task required responses to a subset of the changes. Detection rates at successive points in time were measured by response time (RT) hazard functions.The principal finding was that increasing the set size divided the detection rates-and these divisive effects were essentially constant over time and over the time-varying influence of the target signals and response tasks. The set size, visual target signal, and response task exerted mutually invariant influence on detection rates at given times, indicating independent joint contributions of parallel component processes. The lawful structure of these effects was measured by RT hazard functions but not by RTs as such. The results generalized the time-invariant divisive effects of set size on visual process rates found by Lappin, Morse, & Seiffert (Attention, Perception, & Psychophysics, 78, 2469-2493, 2016). These findings suggest that the rate of visual perception has a limiting channel capacity.

Search for Face Identity or Expression: Set Size Effects in Developmental Prosopagnosia

The set size effect during visual search indexes the effects of processing load and thus the efficiency of perceptual mechanisms. Our goal was to investigate whether individuals with developmental prosopagnosia show increased set size effects when searching faces for face identity and how this compares to search for face expression. We tested 29 healthy individuals and 13 individuals with developmental prosopagnosia. Participants were shown sets of three to seven faces to judge whether the identities or expressions of the faces were the same across all stimuli or if one differed. The set size effect was the slope of the linear regression between the number of faces in the array and the response time. Accuracy was similar in both controls and prosopagnosic participants. Developmental prosopagnosic participants displayed increased set size effects in face identity search but not in expression search. Single-participant analyses reveal that 11 developmental prosopagnosic participants showed a putative classical dissociation, with impairments in identity but not expression search. Signal detection theory analysis showed that identity set size effects were highly reliable in discriminating prosopagnosic participants from controls. Finally, the set size ratios of same to different trials were consistent with the predictions of self-terminated serial search models for control participants and prosopagnosic participants engaged in expression search but deviated from those predictions for identity search by the prosopagnosic cohort. We conclude that the face set size effect reveals a highly prevalent and selective perceptual inefficiency for processing face identity in developmental prosopagnosia.

What is a preattentive feature?

The concept of a preattentive feature has been central to vision and attention research for about half a century. A preattentive feature is a feature that guides attention in visual search and that cannot be decomposed into simpler features. While that definition seems straightforward, there is no simple diagnostic test that infallibly identifies a preattentive feature. This paper briefly reviews the criteria that have been proposed and illustrates some of the difficulties of definition.

Visual search for complex objects: Set-size effects for faces, words and cars

To compare visual processing for different object types, we developed visual search tests that generated accuracy and response time parameters, including an object set-size effect that indexes perceptual processing load. Our goal was to compare visual search for two expert object types, faces and visual words, as well as a less expert type, cars. We first asked if faces and words showed greater inversion effects in search. Second, we determined whether search with upright stimuli correlated with other perceptual indices. Last we assessed for correlations between tests within a single orientation, and between orientations for a single object type. Object set-size effects were smaller for faces and words than cars. All accuracy and temporal measures showed an inversion effect for faces and words, but not cars. Face-search accuracy measures correlated with accuracy on the Cambridge Face Memory Test and word-search temporal measures correlated with single-word reading times, but car search did not correlate with semantic car knowledge. There were cross-orientation correlations for all object types, as well as cross-object correlations in the inverted orientation, while in the upright orientation face search did not correlate with word or car search. We conclude that object search shows effects of expertise. Compared to cars, words and faces showed smaller object set-size effects, greater inversion effects, and their search results correlated with other indices of perceptual expertise. The correlation analyses provide preliminary evidence supporting contributions from common processes in the case of inverted stimuli, object-specific processes that operate in both orientations, and distinct processing for upright faces.

Perceptual learning produces perceptual objects

In their seminal study of chess expertise, Simon and Chase (Chase & Simon, 1973; Simon & Chase, 1973) proposed that perceptual learning was a necessary component of skill acquisition. In their view, acquisition of skill results from the strategic use of learning at multiple levels to adaptively overcome inherent limitations. The knowledge acquired by way of perceptual learning that supported increasingly sophisticated perceptual discrimination processes, according to Simon and Chase, was referred to as a chunk. The chunk was conceptualized as a meaningful complex set of features that abstracted the notion of a perceptual object. Simon and Chase further suggested that meaningful combinations of chunks could be combined to form configurations (Simon & Chase, 1973, p. 399). The present study addresses this idea by framing the notion of a chunk in terms of two formal metatheories, one that addresses representation (Ashby & Townsend, 1986) and one that addresses processing (Townsend & Nozawa, 1995), and tests the prediction that perceptual learning produces organized perceptual objects (chunks). Two experiments combine behavioral and electroencephelographic (EEG) measures to show that perceptual learning produces (a) a shift from perceptual independence and separability to violations of separability, and (b) shifts from limited-capacity serial processing to supercapacity parallel processing. The evidence from both experiments is strong and consistent: perceptual learning does indeed induce chunking-the production of perceptual objects, and the foundation of perceptual expertise. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Introspection during short-term memory scanning

The literature in metacognition has argued for many years that introspective access to our own mental content is restricted to the cognitive states associated with the response to a task, such as the level of confidence in a decision or the estimation of the response time; however, the cognitive processes that underlie such states were deemed inaccessible to participants' consciousness. Here, we ask whether participants could introspectively distinguish the cognitive processes that underlie two short-term memory tasks. For this purpose, we asked participants, on a trial-by-trial basis, to report the number of items that they mentally scanned during their short-term memory retrieval, which we have named "subjective number of scanned items." The subjective number of scanned items index was evaluated, in Experiment 1, immediately after a judgment of recency task and, in Experiment 2, after an item recognition task. Finally, in Experiment 3, both tasks were randomly mixed. The results showed that participants' introspection successfully accessed the complexity of the decisional processes.

Magnetoencephalography decoding reveals structural differences within integrative decision processes

When confronted with complex inputs consisting of multiple elements, humans use various strategies to integrate the elements quickly and accurately. For instance, accuracy may be improved by processing elements one at a time^1-4 or over extended periods^5-8; speed can increase if the internal representation of elements is accelerated^9,10. However, little is known about how humans actually approach these challenges because behavioural findings can be accounted for by multiple alternative process models¹¹ and neuroimaging investigations typically rely on haemodynamic signals that change too slowly. Consequently, to uncover the fast neural dynamics that support information integration, we decoded magnetoencephalographic signals that were recorded as human subjects performed a complex decision task. Our findings reveal three sources of individual differences in the temporal structure of the integration process-sequential representation, partial reinstatement and early computation-each having a dissociable effect on how subjects handled problem complexity and temporal constraints. Our findings shed new light on the structure and influence of self-determined neural integration processes.

The channel capacity of visual awareness divided among multiple moving objects

If attention is distributed among multiple moving objects, how does this divided attention affect the temporal process for detecting a specific target motion? Well-trained observers in three experiments monitored ongoing random motions of multiple objects, trying to rapidly detect non-random target motions. Response time hazard rates revealed a simple lawful structure of the detection processes. Target detection rates (hazard rates, in bits /s) were inversely proportional to the number of observed objects. Detection rates at any response time and in any condition equaled a product of two parallel (functionally independent and concurrent) visual processes: visual awareness and motion integration. The rate of visual awareness was inversely proportional to Set Size (n = 1-12), constant over time, and invariant with integrated motion information. Thus, a single rate parameter, indicating a constant channel capacity of visual awareness, described detection rates over a wide range of conditions and response times. During an initial interval of roughly 0.5 s, detection rates increased proportionally with the duration and length of motion; but after this initial integration, detection rates were constant, independent of the time the target remained undetected. The relationship between the quantity of visual information and detection rates was simpler than anticipated by contemporary theories of attention, perception, and performance.

Five Factors that Guide Attention in Visual Search

How do we find what we are looking for? Fundamental limits on visual processing mean that even when the desired target is in our field of view, we often need to search, because it is impossible to recognize everything at once. Searching involves directing attention to objects that might be the target. This deployment of attention is not random. It is guided to the most promising items and locations by five factors discussed here: Bottom-up salience, top-down feature guidance, scene structure and meaning, the previous history of search over time scales from msec to years, and the relative value of the targets and distractors. Modern theories of search need to specify how all five factors combine to shape search behavior. An understanding of the rules of guidance can be used to improve the accuracy and efficiency of socially-important search tasks, from security screening to medical image perception.

ACC Neuro-over-Connectivity Is Associated with Mathematically Modeled Additional Encoding Operations of Schizophrenia Stroop-Task Performance

Functional magnetic resonance imaging at 7.0 Tesla was undertaken among Schizophrenia participants (Sz), and clinical (major mood disorder; MDD) and healthy controls (HC), during performance of the Stoop task. Stroop conditions included congruent and incongruent word color items, color-only items, and word-only items. Previous modeling results extended to this most widely used selective-attention task. All groups executed item-encoding operations (subprocesses of the item encoding process) at the same rate (performance accuracy being similarly high throughout), thus displaying like processing capacity; Sz participants, however, employed more subprocesses for item completions than did the MDD participants, who in turn used more subprocesses than the HC group. The reduced efficiency in deploying cognitive-workload capacity among the Sz participants was paralleled by more diffuse neuroconnectivity (Blood-Oxygen-Level-Dependent co-activation) with the anterior cingulate cortex (ACC) (Broadman Area 32), spreading away from this encoding-intensive region; and by less evidence of network dissociation across Stroop conditions. Estimates of cognitive work done to accomplish item completion were greater for the Sz participants, as were estimates of entropy in both the modeled trial-latency distribution, and its associated neuro-circuitry. Findings are held to be symptom and assessment significant, and to have potential implications for clinical intervention.

Visual Search Revived: The Slopes Are Not That Slippery: A Reply to Kristjansson (2015)

Kristjansson (2015) suggests that standard research methods in the study of visual search should be “reconsidered.” He reiterates a useful warning against treating reaction time x set size functions as simple metrics that can be used to label search tasks as “serial” or “parallel.” However, I argue that he goes too far with a broad attack on the use of slopes in the study of visual search. Used wisely, slopes do provide us with insight into the mechanisms of visual search.

Position tracking and identity tracking are separate systems: Evidence from eye movements

How do we track multiple moving objects in our visual environment? Some investigators argue that tracking is based on a parallel mechanism (e.g., Cavanagh & Alvarez, 2005; Pylyshyn, 1989), others argue that tracking contains a serial component (e.g. Holcombe & Chen, 2013; Oksama & Hyönä, 2008). In the present study, we put previous theories into a direct test by registering observers' eye movements when they tracked identical moving targets (the MOT task) or when they tracked distinct object identities (the MIT task). The eye movement technique is a useful tool to study whether overt focal attention is exploited during tracking. We found a qualitative difference between these tasks in terms of eye movements. When the participants tracked only position information (MOT), the observers had a clear preference for keeping their eyes fixed for a rather long time on the same screen position. In contrast, active eye behavior was observed when the observers tracked the identities of moving objects (MIT). The participants updated over four target identities with overt attention shifts. These data suggest that there are two separate systems involved in multiple object tracking. The position tracking system keeps track of the positions of the moving targets in parallel without the need of overt attention shifts in the form of eye movements. On the other hand, the identity tracking system maintains identity-location bindings in a serial fashion by utilizing overt attention shifts.

Guided search for triple conjunctions

A key tenet of feature integration theory and of related theories such as guided search (GS) is that the binding of basic features requires attention. This would seem to predict that conjunctions of features of objects that have not been attended should not influence search. However, Found (1998) reported that an irrelevant feature (size) improved the efficiency of search for a Color × Orientation conjunction if it was correlated with the other two features across the display, as compared to the case in which size was not correlated with color and orientation features. We examined this issue with somewhat different stimuli. We used triple conjunctions of color, orientation, and shape (e.g., search for a red, vertical, oval-shaped item). This allowed us to manipulate the number of features that each distractor shared with the target (sharing) and it allowed us to vary the total number of distractor types (and, thus, the number of groups of identical items: grouping). We found that these triple conjunction searches were generally very efficient--producing very shallow Reaction Time × Set Size slopes, consistent with strong guidance by basic features. Nevertheless, both of the variables, sharing and grouping, modulated performance. These influences were not predicted by previous accounts of GS; however, both can be accommodated in a GS framework. Alternatively, it is possible, though not necessary, to see these effects as evidence for "preattentive binding" of conjunctions.

Content-specific evidence accumulation in inferior temporal cortex during perceptual decision-making

During a perceptual decision, neuronal activity can change as a function of time-integrated evidence. Such neurons may serve as decision variables, signaling a choice when activity reaches a boundary. Because the signals occur on a millisecond timescale, translating to human decision-making using functional neuroimaging has been challenging. Previous neuroimaging work in humans has identified patterns of neural activity consistent with an accumulation account. However, the degree to which the accumulating neuroimaging signals reflect specific sources of perceptual evidence is unknown. Using an extended face/house discrimination task in conjunction with cognitive modeling, we tested whether accumulation signals, as measured using functional magnetic resonance imaging (fMRI), are stimulus-specific. Accumulation signals were defined as a change in the slope of the rising edge of activation corresponding with response time (RT), with higher slopes associated with faster RTs. Consistent with an accumulation account, fMRI activity in face- and house-selective regions in the inferior temporal cortex increased at a rate proportional to decision time in favor of the preferred stimulus. This finding indicates that stimulus-specific regions perform an evidence integrative function during goal-directed behavior and that different sources of evidence accumulate separately. We also assessed the decision-related function of other regions throughout the brain and found that several regions were consistent with classifications from prior work, suggesting a degree of domain generality in decision processing. Taken together, these results provide support for an integration-to-boundary decision mechanism and highlight possible roles of both domain-specific and domain-general regions in decision evidence evaluation.

Part-whole information assists in topological × topological but not in orientation × orientation conjunction searches

Visual search is a routine task used in everyday life and is an important field of research in cognitive psychology. In laboratory settings, it has been shown that search for a target defined by a unique conjunction of two colours is more efficient if one colour surrounds the other (a part-whole search) compared to when no such hierarchical structural relationship exists (a part-part search; Wolfe et al. in Perception & Psychophysics, 55, 537, 1994). A similar result has been shown to hold for size × size conjunction searches (Bilsky & Wolfe in Perception & Psychophysics, 57, 749, 1995). We show that this result also holds for topology × topology conjunction searches (where the stimuli are either hollow or filled), but not for orientation × orientation conjunction searches. We use the simultaneous-sequential paradigm to investigate a possible reason for the inefficiency of part-whole orientation search compared with the efficiency of part-whole searches of other features. We argue that two different attribute values from the same dimension can be processed independently, without interfering with each other for colour, size, and topology, but not for orientation. Because it is obviously more efficient to process a conjunction stimulus when both components of the conjunction can be processed without mutual interference, it follows that colour × colour, size × size, and topological × topology part-whole conjunction searches are likely to be more efficient than orientation × orientation part-whole conjunction searches.

Optional Stopping and the Termination of Memory Retrieval

Recent years have seen an increased interest in understanding memory-retrieval dynamics and, in particular, what makes a person decide to terminate the memory-search process. We review research that has employed the open-ended retrieval paradigm (Dougherty & Harbison, 2007) and focus on the behavioral regularities it has revealed. The main finding of this research is that people’s memory-search behavior follows a lawful pattern of a convex decreasing relation between exit latency, or the time between the final retrieval and the decision to terminate search, and the number of items retrieved. Theoretical work has converged on a stopping rule that treats the retrieval process as a costly cognitive process that is truncated on the basis of a comparative judgment of perceived relative benefit. Parallels with other search domains are highlighted.

Introduction to the special issue on visual working memory

Visual working memory is a volatile, limited-capacity memory that appears to play an important role in our impression of a visual world that is continuous in time. It also mediates between the contents of the mind and the contents of that visual world. Research on visual working memory has become increasingly prominent in recent years. The articles in this special issue of Attention, Perception, & Psychophysics describe new empirical findings and theoretical understandings of the topic.

Bilateral and two-item advantage in subitizing

Subitizing refers to people's ability to enumerate small sets of items fast and accurately. The present study examined if the speed and scope of subitizing is improved when the items to be enumerated are presented bilaterally across hemifields rather than unilaterally in a single hemifield. Such an effect, known as the bilateral field advantage, has been observed in a number of other visual tasks. A second aim was to examine whether the speed of subitizing could be explained by the speed it takes to detect the items to be enumerated, as simple reaction times to multiple stimuli are known to be faster than responses to individual items (known as the redundant target effect, RTE). The results revealed a bilateral field advantage even for enumerating two items. Moreover, the two item condition was the optimal subitizing condition - even enumerating one single item took longer - but this effect was not due to the RTE. In fact, the RTE negatively correlated with the speed of enumerating the same stimuli.

A new perspective on binaural integration using response time methodology: super capacity revealed in conditions of binaural masking release

This study applied reaction-time based methods to assess the workload capacity of binaural integration by comparing reaction time (RT) distributions for monaural and binaural tone-in-noise detection tasks. In the diotic contexts, an identical tone + noise stimulus was presented to each ear. In the dichotic contexts, an identical noise was presented to each ear, but the tone was presented to one of the ears 180° out of phase with respect to the other ear. Accuracy-based measurements have demonstrated a much lower signal detection threshold for the dichotic vs. the diotic conditions, but accuracy-based techniques do not allow for assessment of system dynamics or resource allocation across time. Further, RTs allow comparisons between these conditions at the same signal-to-noise ratio. Here, we apply a reaction-time based capacity coefficient, which provides an index of workload efficiency and quantifies the resource allocations for single ear vs. two ear presentations. We demonstrate that the release from masking generated by the addition of an identical stimulus to one ear is limited-to-unlimited capacity (efficiency typically less than 1), consistent with less gain than would be expected by probability summation. However, the dichotic presentation leads to a significant increase in workload capacity (increased efficiency)-most specifically at lower signal-to-noise ratios. These experimental results provide further evidence that configural processing plays a critical role in binaural masking release, and that these mechanisms may operate more strongly when the signal stimulus is difficult to detect, albeit still with nearly 100% accuracy.

The role of partial knowledge in statistical word learning

A critical question about the nature of human learning is whether it is an all-or-none or a gradual, accumulative process. Associative and statistical theories of word learning rely critically on the later assumption: that the process of learning a word's meaning unfolds over time. That is, learning the correct referent for a word involves the accumulation of partial knowledge across multiple instances. Some theories also make an even stronger claim: partial knowledge of one word-object mapping can speed up the acquisition of other word-object mappings. We present three experiments that test and verify these claims by exposing learners to two consecutive blocks of cross-situational learning, in which half of the words and objects in the second block were those that participants failed to learn in Block 1. In line with an accumulative account, Re-exposure to these mis-mapped items accelerated the acquisition of both previously experienced mappings and wholly new word-object mappings. But how does partial knowledge of some words speed the acquisition of others? We consider two hypotheses. First, partial knowledge of a word could reduce the amount of information required for it to reach threshold, and the supra-threshold mapping could subsequently aid in the acquisition of new mappings. Alternatively, partial knowledge of a word's meaning could be useful for disambiguating the meanings of other words even before the threshold of learning is reached. We construct and compare computational models embodying each of these hypotheses and show that the latter provides a better explanation of the empirical data.

Neural dynamics of audiovisual speech integration under variable listening conditions: an individual participant analysis

Speech perception engages both auditory and visual modalities. Limitations of traditional accuracy-only approaches in the investigation of audiovisual speech perception have motivated the use of new methodologies. In an audiovisual speech identification task, we utilized capacity (Townsend and Nozawa, 1995), a dynamic measure of efficiency, to quantify audiovisual integration. Capacity was used to compare RT distributions from audiovisual trials to RT distributions from auditory-only and visual-only trials across three listening conditions: clear auditory signal, S/N ratio of −12 dB, and S/N ratio of −18 dB. The purpose was to obtain EEG recordings in conjunction with capacity to investigate how a late ERP co-varies with integration efficiency. Results showed efficient audiovisual integration for low auditory S/N ratios, but inefficient audiovisual integration when the auditory signal was clear. The ERP analyses showed evidence for greater audiovisual amplitude compared to the unisensory signals for lower auditory S/N ratios (higher capacity/efficiency) compared to the high S/N ratio (low capacity/inefficient integration). The data are consistent with an interactive framework of integration, where auditory recognition is influenced by speech-reading as a function of signal clarity.

Serial consolidation of orientation information into visual short-term memory

Previous research suggests that there is a limit to the rate at which items can be consolidated in visual short-term memory (VSTM). This limit could be due to either a serial or a limited-capacity parallel process. Historically, it has proven difficult to distinguish between these two types of processes. In the present experiment, we took a novel approach that allowed us to do so. Participants viewed two oriented gratings either sequentially or simultaneously and reported one of the gratings' orientation via method of adjustment. Performance was worse for the simultaneous than for the sequential condition. We fit the data with a mixture model that assumes performance is limited by a noisy memory representation plus random guessing. Critically, the serial and limited-capacity parallel processes made distinct predictions regarding the model's guessing and memory-precision parameters. We found strong support for a serial process, which implies that one can consolidate only a single orientation into VSTM at a time.

When is it time to move to the next raspberry bush? Foraging rules in human visual search

Animals, including humans, engage in many forms of foraging behavior in which resources are collected from the world. This paper examines human foraging in a visual search context. A real-world analog would be berry picking. The selection of individual berries is not the most interesting problem in such a task. Of more interest is when does a forager leave one patch or berry bush for the next one? Marginal Value Theorem (MVT; Charnov, 1976) predicts that observers will leave a patch when the instantaneous yield from that patch drops below the average yield from the entire "field." Experiments 1, 2, 3, and 4 show that MVT gives a good description of human behavior for roughly uniform collections of patches. Experiments 5 and 6 show strong departures from MVT when patch quality varies and when visual information is degraded.

Limited featured-based attention to multiple features

Attending to a feature (e.g., color or motion direction) can enhance the early visual processing of that feature. However, it is not known whether one can simultaneously enhance multiple features. We examined people's ability to attend to multiple features in a feature cueing paradigm. Each trial contained two intervals consisting of a random dot motion stimulus. One interval (noise) had 0% coherence (no net motion), while the other interval (signal) moved in a particular direction with varying levels of coherence. Participants reported which interval contained the signal in one of three cueing conditions. In the one-cue condition, a line segment preceded the stimuli indicating the direction of the signal with 100% validity. In the two-cue condition, two lines preceded the stimuli, indicating the signal would move in one of the two cued directions. In the no-cue condition, no line segment appeared before the dot stimuli. In several experiments, we consistently observed a lower detection threshold in the one-cue condition than the no-cue condition, showing that participants can enhance processing of a single feature. However, detection threshold was consistently higher for the two-cue than one-cue condition, indicating that participants could not simultaneously enhance two motion directions as effectively as one direction. This finding revealed a severe capacity limit in our ability to enhance early visual processing for multiple features.

Signal detection evidence for limited capacity in visual search

The nature of capacity limits (if any) in visual search has been a topic of controversy for decades. In 30 years of work, researchers have attempted to distinguish between two broad classes of visual search models. Attention-limited models have proposed two stages of perceptual processing: an unlimited-capacity preattentive stage, and a limited-capacity selective attention stage. Conversely, noise-limited models have proposed a single, unlimited-capacity perceptual processing stage, with decision processes influenced only by stochastic noise. Here, we use signal detection methods to test a strong prediction of attention-limited models. In standard attention-limited models, performance of some searches (feature searches) should only be limited by a preattentive stage. Other search tasks (e.g., spatial configuration search for a "2" among "5"s) should be additionally limited by an attentional bottleneck. We equated average accuracies for a feature and a spatial configuration search over set sizes of 1-8 for briefly presented stimuli. The strong prediction of attention-limited models is that, given overall equivalence in performance, accuracy should be better on the spatial configuration search than on the feature search for set size 1, and worse for set size 8. We confirm this crossover interaction and show that it is problematic for at least one class of one-stage decision models.

A new approach for assessment of mental architecture: repeated tagging

A new approach to the study of a relatively neglected property of mental architecture-whether and when the already-processed elements are separated from the to-be-processed elements-is proposed. The process of numerical proportion discrimination between two sets of elements defined either by color or by orientation can be described as sampling with or without replacement (characterized by binomial or hypergeometric probability distributions respectively) depending on the possibility to tag an element once or repeatedly. All empirical psychometric functions were approximated by a theoretical model showing that the ability to keep track of the already tagged elements is not an inflexible part of the mental architecture but rather an individually variable strategy which also depends on conspicuity of perceptual attributes. Strong evidence is provided that in a considerable number of trials, observers tagged the same element repeatedly which can only be done serially at two separate time moments.

Visual search for arbitrary objects in real scenes

How efficient is visual search in real scenes? In searches for targets among arrays of randomly placed distractors, efficiency is often indexed by the slope of the reaction time (RT) × Set Size function. However, it may be impossible to define set size for real scenes. As an approximation, we hand-labeled 100 indoor scenes and used the number of labeled regions as a surrogate for set size. In Experiment 1, observers searched for named objects (a chair, bowl, etc.). With set size defined as the number of labeled regions, search was very efficient (~5 ms/item). When we controlled for a possible guessing strategy in Experiment 2, slopes increased somewhat (~15 ms/item), but they were much shallower than search for a random object among other distinctive objects outside of a scene setting (Exp. 3: ~40 ms/item). In Experiments 4-6, observers searched repeatedly through the same scene for different objects. Increased familiarity with scenes had modest effects on RTs, while repetition of target items had large effects (>500 ms). We propose that visual search in scenes is efficient because scene-specific forms of attentional guidance can eliminate most regions from the "functional set size" of items that could possibly be the target.

Additive factors do not imply discrete processing stages: a worked example using models of the stroop task

Previously, it has been shown experimentally that the psychophysical law known as Piéron's Law holds for color intensity and that the size of the effect is additive with that of Stroop condition (Stafford et al., 2011). According to the additive factors method (Donders, 1868-1869/1969; Sternberg, 1998), additivity is assumed to indicate independent and discrete processing stages. We present computational modeling work, using an existing Parallel Distributed Processing model of the Stroop task (Cohen et al., 1990) and a standard model of decision making (Ratcliff, 1978). This demonstrates that additive factors can be successfully accounted for by existing single stage models of the Stroop effect. Consequently, it is not valid to infer either discrete stages or separate loci of effects from additive factors. Further, our modeling work suggests that information binding may be a more important architectural property for producing additive factors than discrete stages.

Some behavioral and neurobiological constraints on theories of audiovisual speech integration: a review and suggestions for new directions

Summerfield (1987) proposed several accounts of audiovisual speech perception, a field of research that has burgeoned in recent years. The proposed accounts included the integration of discrete phonetic features, vectors describing the values of independent acoustical and optical parameters, the filter function of the vocal tract, and articulatory dynamics of the vocal tract. The latter two accounts assume that the representations of audiovisual speech perception are based on abstract gestures, while the former two assume that the representations consist of symbolic or featural information obtained from visual and auditory modalities. Recent converging evidence from several different disciplines reveals that the general framework of Summerfield's feature-based theories should be expanded. An updated framework building upon the feature-based theories is presented. We propose a processing model arguing that auditory and visual brain circuits provide facilitatory information when the inputs are correctly timed, and that auditory and visual speech representations do not necessarily undergo translation into a common code during information processing. Future research on multisensory processing in speech perception should investigate the connections between auditory and visual brain regions, and utilize dynamic modeling tools to further understand the timing and information processing mechanisms involved in audiovisual speech integration.

An assessment of behavioral dynamic information processing measures in audiovisual speech perception

Research has shown that visual speech perception can assist accuracy in identification of spoken words. However, little is known about the dynamics of the processing mechanisms involved in audiovisual integration. In particular, architecture and capacity, measured using response time methodologies, have not been investigated. An issue related to architecture concerns whether the auditory and visual sources of the speech signal are integrated “early” or “late.” We propose that “early” integration most naturally corresponds to coactive processing whereas “late” integration corresponds to separate decisions parallel processing. We implemented the double factorial paradigm in two studies. First, we carried out a pilot study using a two-alternative forced-choice discrimination task to assess architecture, decision rule, and provide a preliminary assessment of capacity (integration efficiency). Next, Experiment 1 was designed to specifically assess audiovisual integration efficiency in an ecologically valid way by including lower auditory S/N ratios and a larger response set size. Results from the pilot study support a separate decisions parallel, late integration model. Results from both studies showed that capacity was severely limited for high auditory signal-to-noise ratios. However, Experiment 1 demonstrated that capacity improved as the auditory signal became more degraded. This evidence strongly suggests that integration efficiency is vitally affected by the S/N ratio.

Extending the simultaneous-sequential paradigm to measure perceptual capacity for features and words

In perception, divided attention refers to conditions in which multiple stimuli are relevant to an observer. To measure the effect of divided attention in terms of perceptual capacity, we introduce an extension of the simultaneous-sequential paradigm. The extension makes predictions for fixed-capacity models as well as for unlimited-capacity models. We apply this paradigm to two example tasks, contrast discrimination and word categorization, and find dramatically different effects of divided attention. Contrast discrimination has unlimited capacity, consistent with independent, parallel processing. Word categorization has a nearly fixed capacity, consistent with either serial processing or fixed-capacity, parallel processing. We argue that these measures of perceptual capacity rely on relatively few assumptions compared to most alternative measures.