Recognition of letters displayed as briefly flashed dot patterns

Human perception of flicker-fused letters that are luminance balanced

The Talbot-Plateau law specifies what combinations of flash frequency, duration, and intensity will yield a flicker-fused stimulus that matches the brightness of a steady stimulus. It has proven to be remarkably robust in its predictions, and here we provide additional support though the use of a contrast discrimination task. However, we also find that the visual system can register flicker-fused letters when the combination of frequency and duration is relatively low. The letters are recognized even though they have the same physical luminance as background. We hypothesize that the letters elicit synchronous oscillations that encode for stimulus attributes, which prevents the letter from blending into the background.

Evaluating integration of letter fragments through contrast and spatially targeted masking

Four experiments were conducted to gain a better understanding of the visual mechanisms related to how integration of partial shape cues provides for recognition of the full shape. In each experiment, letters formed as outline contours were displayed as a sequence of adjacent segments (fragments), each visible during a 17-ms time frame. The first experiment varied the contrast of the fragments. There were substantial individual differences in contrast sensitivity, so stimulus displays in the masking experiments that followed were calibrated to the sensitivity of each participant. Masks were displayed either as patterns that filled the entire screen (full field) or as successive strips that were sliced from the pattern, each strip lying across the location of the letter fragment that had been shown a moment before. Contrast of masks were varied to be lighter or darker than the letter fragments. Full-field masks, whether light or dark, provided relatively little impairment of recognition, as was the case for mask strips that were lighter than the letter fragments. However, dark strip masks proved to be very effective, with the degree of recognition impairment becoming larger as mask contrast was increased. A final experiment found the strip masks to be most effective when they overlapped the location where the letter fragments had been shown a moment before. They became progressively less effective with increased spatial separation from that location. Results are discussed with extensive reference to potential brain mechanisms for integrating shape cues.

There is a fundamental, unbridgeable gap between DNNs and the visual cortex

Deep neural networks (DNNs) are not just inadequate models of the visual system but are so different in their structure and functionality that they are not even on the same playing field. DNN units have almost nothing in common with neurons, and, unlike visual neurons, they are often fully connected. At best, DNNs can label inputs, while our object perception is both holistic and detail preserving. A feat that no computational system can achieve.

Recognition of letters displayed as successive contour fragments

Shapes can be displayed as parts but perceived as a whole through feedforward and feedback mechanisms in the visual system, though the exact spatiotemporal relationships for this process are still unclear. Our experiments examined the integration of letter fragments that were displayed as a rapid sequence. We examined the effects of timing and masking on integration, hypothesizing that increasing the timing interval between frames would impair recognition by disrupting contour linkage. We further used different mask types, a full-field pattern mask and a smaller strip mask, to examine the effects of global vs local masking on integration. We found that varying mask types and contrast produced a greater decline in recognition than was found when persistence or mask density was manipulated. The study supports prior work on letter recognition and provides greater insight into the spatiotemporal factors that contribute to the identification of shapes.

Anatomical, physiological, and psychophysical data show that the nature of conscious perception is incompatible with the integrated information theory (IIT)

The integrated information theory (IIT) equates levels of consciousness with the amount of information integrated over the elements that constitute a system. Conscious visual perception provides two observations that contradict the IIT. First, objects are accurately perceived when presented for ≪100 ms during which time no neural integration is possible. Second, an object is seen as an integrated whole and, concurrently, all constituent elements are evident. Because integration destroys information about details, IIT cannot account for perceptual detail preservation.

The perceived present: What is it, and what is it there for?

It is proposed that the perceived present is not a moment in time, but an information structure comprising an integrated set of products of perceptual processing. All information in the perceived present carries an informational time marker identifying it as "present". This marker is exclusive to information in the perceived present. There are other kinds of time markers, such as ordinality ("this stimulus occurred before that one") and duration ("this stimulus lasted for 50 ms"). These are different from the "present" time marker and may be attached to information regardless of whether it is in the perceived present or not. It is proposed that the perceived present is a very short-term and very high-capacity holding area for perceptual information. The maximum holding time for any given piece of information is ~100 ms: This is affected by the need to balance the value of informational persistence for further processing against the problem of obsolescence of the information. The main function of the perceived present is to facilitate access by other specialized, automatic processes.

Evaluating spatiotemporal integration of shape cues

Prior work has shown that humans can successfully identify letters that are constructed with a sparse array of dots, wherein the dot pattern reflects the strokes that would normally be used to fashion a given letter. In the present work the dots were briefly displayed, one at a time in sequence, varying the spatial order in which they were shown. A forward sequence was spatially ordered as though one were passing a stroke across the dots to connect them. Experiments compared this baseline condition to the following three conditions: a) the dot sequence was spatially ordered, but in the reverse direction from how letter strokes might normally be written; b) the dots in each stroke of the letter were displayed in a random order; c) the sequence of displayed dots were chosen for display from any location in the letter. Significant differences were found between the baseline condition and all three of the comparison conditions, with letter recognition being far worse for the random conditions than for conditions that provided consistent spatial ordering of dot sequences. These findings show that spatial order is critical for integration of shape cues that have been sequentially displayed.

Binary vs. continuous experimental designs for the study of unconscious perceptual processing

Binary vs. continuous conceptualizations of consciousness may have an unstated influence on experimental designs in unconscious perception research. The binary approach aims to compare a conscious condition (e.g., supraliminal, no or weak stimulus masking) to an unconscious condition (e.g., subliminal, heavy stimulus masking). In contrast, continuous designs tend to vary stimulus energy along a near-threshold continuum to determine changes in perception as a function of stimulus energy (or duration). The present study compared two experimental designs, binary vs. continuous, for the influence of target-masked prime stimuli on a Stroop task. The display parameters were inspired by emotional Stroop studies reporting unconscious perception. Neither experiment produced strong evidence of unconscious perception, but the experiment with a continuous design was more informative. We thus recommend sampling a range of near-threshold display parameters to yield straight-forward, unambiguous interpretations.

Spatiotemporal integration of visual stimuli and its relevance to the use of a divisional power supply scheme for retinal prosthesis

A wireless photovoltaic retinal prosthesis is currently being studied with the aim of providing prosthetic vision to patients with retinitis pigmentosa (RP) and age-related macular degeneration (AMD). The major challenge of a photovoltaic device is its limited power efficiency. Our retinal prosthetic design implements a unique divisional power supply scheme (DPSS) system that provides the electrical power generated by all of the solar cells to only a subset of electrodes at any moment in time. The aim of the present study was to systematically characterize the spatiotemporal integration performance of the system under various DPSS conditions using human subjects and a psychophysical approach. A 16x16 pixels LED array controlled by Arduino was used to simulate the output signal of the DPSS design, and human performance under different visual stimulations at various update frequencies was then used to assess the spatiotemporal capability of retinal prostheses. The results showed that the contrast polarity of the image, image brightness, and division number influenced the lower limit of the update frequency of the DPSS system, while, on the other hand, visual angle, ambient light level, and stimulation order did not affect performance significantly. Pattern recognition by visual persistence with spatiotemporal integration of multiple frames of sparse dots is a feasible approach in retinal prosthesis design. These findings provide an insight into how to optimize a photovoltaic retinal prosthesis using a DPSS design with an appropriate update frequency for reliable pattern recognition. This will help the development of a wireless device able to restore vision to RP and AMD patients in the future.

Masking the Integration of Complementary Shape Cues

Retinal and cortical mechanisms provide for persistence of visual information across intervals of many hundreds of milliseconds, which supports the integration of partial shape cues. The present experiments displayed unknown shapes in a match recognition task, wherein a target shape was quickly followed by a comparison shape; the task was to specify whether the comparison shape was the same or different from the target. The target and comparison shapes were displayed as sparse dots that marked boundary locations. The first experiment successively displayed the target shape as two complementary subsets and found that the probability of correct match remained above chance with up to 500 ms of subset separation. The second experiment demonstrated masking of the target by a random pattern of dots when the target and mask were displayed simultaneously, but with much less or no masking when the two were separated by 100 ms. The third experiment displayed the target subsets with 200 ms of separation and found that match recognition was disrupted when the random-dot mask was displayed midway between the two subsets. Much less masking of an intact target was produced with that amount of temporal separation, which suggests that mechanisms for integration of shape cues have a special vulnerability to masking. The third experiment also found very little impairment of match recognition when the mask was displayed simultaneous with one of the subsets. We hypothesize that there is embedding of the subset pattern within the mask pattern, but additional display of the other subset effectively disembeds the buried partial shape cues.

Very small faces are easily discriminated under long and short exposure times

Acuity measures related to overall face size that can be perceived have not been studied quantitatively. Consequently, experimenters use a wide range of sizes (usually large) without always providing a rationale for their choices. I studied thresholds for face discrimination by presenting both long (500 ms)- and short (17, 33, 50 ms)-duration stimuli. Face width threshold for the long presentation was ~0.2°, and thresholds for the flashed stimuli ranged from ~0.3° for the 17-ms flash to ~0.23° for the 33- and 50-ms flashes. Such thresholds indicate that face stimuli used in physiological or psychophysical experiments are often too large to tap human fine spatial capabilities, and thus interpretations of such experiments should take into account face discrimination acuity. The 0.2° threshold found in this study is incompatible with the prevalent view that faces are represented by a population of specialized “face cells” because those cells do not respond to <1° stimuli and are optimally tuned to >4° faces. Also, the ability to discriminate small, high-spatial frequency flashed face stimuli is inconsistent with models suggesting that fixational drift transforms retinal spatial patterns into a temporal code. It seems therefore that the small image motions occurring during fixation do not disrupt our perception, because all relevant processing is over with before those motions can have significant effects.

NEW & NOTEWORTHY Although face perception is central to human behavior, the minimally perceived face size is not known. This study shows that humans can discriminate very small (~0.2°) faces. Furthermore, even when flashed for tens of milliseconds, ~0.25° faces can be discriminated. Such fine acuity should impact modeling of physiological mechanisms of face perception. The ability to discriminate flashed faces where there is almost no eye movement indicates that eye drift is not essential for visibility.

Evaluating persistence of shape information using a matching protocol

Many laboratories have studied persistence of shape information, the goal being to better understand how the visual system mediates recognition of objects. Most have asked for recognition of known shapes, e.g., letters of the alphabet, or recall from an array. Recognition of known shapes requires access to long-term memory, so it is not possible to know whether the experiment is assessing short-term encoding and working memory mechanisms, or has encountered limitations on retrieval from memory stores. Here we have used an inventory of unknown shapes, wherein a string of discrete dots forms the boundary of each shape. Each was displayed as a target only once to a given respondent, with recognition being tested using a matching task. Analysis based on signal detection theory was used to provide an unbiased estimate of the probability of correct decisions about whether comparison shapes matched target shapes. Four experiments were conducted, which found the following: a) Shapes were identified with a high probability of being correct with dot densities ranging from 20% to 4%. Performance dropped only about 10% across this density range. b) Shape identification levels remained very high with up to 500 milliseconds of target and comparison shape separation. c) With one-at-a-time display of target dots, varying the total time for a given display, the proportion of correct decisions dropped only about 10% even with a total display time of 500 milliseconds. d) With display of two complementary target subsets, also varying the total time of each display, there was a dramatic decline of proportion correct that reached chance levels by 500 milliseconds. The greater rate of decline for the two-pulse condition may be due to a mechanism that registers when the number of dots is sufficient to create a shape summary. Once a summary is produced, the temporal window that allows shape information to be added may be more limited.

Visual dot interaction with short-term memory

Aim: Many neurodegenerative diseases have a memory component. Brain structures related to memory are affected by environmental stimuli, and it is difficult to dissociate effects of all behavior of neurons. Materials & methods: Here, visual cortex of mice was stimulated with gratings and dot, and an observation of neuronal activity before and after was made. Bandwidth, firing rate and orientation selectivity index were evaluated. Results: A primary communication between primary visual cortex and short-term memory appeared to show an interesting path to train cognitive circuitry and investigate the basics mechanisms of the neuronal learning. The findings also suggested the interplay between primary visual cortex and short-term plasticity. Conclusion: The properties inside a visual target shape the perception and affect the basic encoding. Using visual cortex, it may be possible to train the memory and improve the recovery of people with cognitive disabilities or memory deficit.

Information persistence evaluated with low-density dot patterns

After more than a century of study, we do not yet fully understand how shapes and patterns are encoded and identified. Greater progress might result from quantifying stimulus information, thus allowing manipulation of the degree to which a shape or pattern can elicit recognition. The present work used discrete dot patterns that are seen as letters of the alphabet. By adjusting the density of the dots in each pattern, one can determine the probability that it will be recognized. The experiments displayed low-density dot patterns to human respondents, assessing the interval across which non-redundant information provided by two compatible subsets would combine to elicit recognition. This provided a measure of the time required for decay of information persistence. Viewed in the context of prior work, the evidence indicates that the retina mediates initial visibility of the stimulus trace, but the longer-duration persistence required for memory retrieval is mediated by visual cortex.

Recognizing Words and Reading Sentences with Microsecond Flash Displays

Strings of dots can be used to construct easily identifiable letters, and these in turn can be used to write words and sentences. Prior work found that respondents could identify individual letters when all the dots were simultaneously flashed for an ultra-brief duration. Four of the experiments reported here constructed five-letter words with these dot-letters and a fifth experiment used them to write complete sentences. Respondents were able to recognize individual words that were displayed with a single, simultaneous ultra-brief flash of all the letters. Further, sentences could be efficiently read with a sequence of simultaneous flashes at a frequency that produced perceptual fusion. One experiment determined the frequency range that would produce flicker-fusion. Two experiments established the relation of intensity to probability of recognition with single flashes and with fused-flicker frequencies. Another established the intensities at which flicker-fused and steady displays were judged to be equal in brightness. The final experiment used those flicker-fused and steady intensities to display sentences. The two display conditions were read with equal efficiency, even though the flicker-fused displays provided light stimulation only 0.003% of the time.

Consciousness weaves our internal view of the outside world

Low-level consciousness is fundamental to our understanding of the world. Within the conscious field, the constantly changing external visual information is transformed into stable, object-based percepts. Remarkably, holistic objects are perceived while we are cognizant of all of the spatial details comprising the objects and of the relationship between individual elements. This parallel conscious association is unique to the brain. Conscious contributions to motor activity come after our understanding of the world has been established.