Are we underestimating the richness of visual experience?

Consciousness Under the Spotlight: The Problem of Measuring Subjective Experience

The study of consciousness is considered by many one of the most difficult contemporary scientific endeavors and confronts several methodological and theoretical challenges. A central issue that makes the study of consciousness so challenging is that, while the rest of science is concerned with problems that can be verified from a "third person" view (i.e., objectively), the study of consciousness deals with the phenomenon of subjective experience, only accessible from a "first person" view. In the present article, we review early (starting during the late 19th century) and later efforts on measuring consciousness and its absence, focusing on the two main approaches used by researchers within the field: objective (i.e., performance based) and subjective (i.e., report based) measures of awareness. In addition, we compare the advantages and disadvantages of both types of awareness measures, evaluate them according to different methodological considerations, and discuss, among other issues, the possibility of comparing them by transforming them to a common sensitivity measure (d'). Finally, we explore several new approaches-such as Bayesian models to support the absence of awareness or new machine-learning based decoding models-as well as future challenges-such as measuring the qualia, the qualitative contents of awareness-in consciousness research.

More than words: can free reports adequately measure the richness of perception?

The question of the richness (or sparseness) of conscious experience has evoked ongoing debate and discussion. Claims for both richness and sparseness are supported by empirical data, yet they are often indirect, and alternative explanations have been put forward. Recently, it has been suggested that current experimental methods limit participants' responses, thereby preventing researchers from assessing the actual richness of perception. Instead, free verbal reports were presented as a possible way to overcome this limitation. As part of this approach, a novel paradigm of freely reported words was developed using a new metric, intersubjective agreement (IA), with experimental results interpreted as capturing aspects of conscious perception. Here, we challenge the validity of freely reported words as a tool for studying the richness of conscious experience. We base our claims on two studies (each composed of three experiments), where we manipulated the richness of percepts and tested whether IA changed accordingly. Five additional control experiments were conducted to validate the experimental logic and examine alternative explanations. Our results suggest otherwise, presenting four challenges to the free verbal report paradigm: first, impoverished stimuli did not evoke lower IA scores. Second, the IA score was correlated with word frequency in English. Third, the original positive relationship between IA scores and rated confidence was not found in any of the six experiments. Fourth, a high rate of nonexisting words was found, some of which described items that matched the gist of the scene but did not appear in the image. We conclude that a metric based on freely reported words might be better explained by vocabulary conventions and gist-based reports than by capturing the richness of perception.

Familiarity Alters the Bandwidth of Perceptual Awareness

Results from paradigms like change blindness and inattentional blindness indicate that observers are unaware of numerous aspects of the visual world. However, intuition suggests that perceptual experience is richer than these results indicate. Why does it feel like we see so much when the data suggests we see so little? One possibility stems from the fact that experimental studies always present observers with stimuli that they have never seen before. Meanwhile, when forming intuitions about perceptual experience, observers reflect on their experiences with scenes with which they are highly familiar (e.g., their office). Does prior experience with a scene change the bandwidth of perceptual awareness? Here, we asked if observers were better at noticing alterations to the periphery in familiar scenes compared with unfamiliar scenes. We found that observers noticed changes to the periphery more frequently with familiar stimuli. Signal detection theoretic analyses revealed that when observers are unfamiliar with a stimulus, they are less sensitive at noticing (d') and are more conservative in their response criterion (c). Taken together, these results suggest that prior knowledge expands the bandwidth of perceptual awareness. It should be stressed that these results challenge the widely held idea that prior knowledge fills in perception. Overall, these findings highlight how prior knowledge plays an important role in determining the limits of perceptual experience and is an important factor to consider when attempting to reconcile the tension between empirical observation and personal introspection.

An embarrassment of richnesses: the PFC isn't the content NCC

Recent years have seen the rise of several theories saying that the prefrontal cortex (PFC) is a neural correlate of visual consciousness (NCC). Especially popular here are theories saying that the PFC is the 'content NCC' for vision, i.e. it contains those brain areas that are not only necessary for consciousness, but also determine 'what' it is that we visually experience (e.g. whether we experience green or red). This article points out how this "upper-deck" form of PFC theory is at odds with the character of visual experience: on the one hand, visual consciousness appears to contain copious amounts of content, with many properties (such as object, shape, or color) being simultaneously represented in many parts of the visual field. On the other hand, the functions that the PFC carries out (e.g. attention and working memory) are each dedicated to processing only a relatively small subset of available visual stimuli. In short, the PFC probably does not produce enough or the right kind of visual representations for it to supply all of the content found in visual experience, in which case the idea that the PFC is the content NCC for vision is probably false. This article also discusses data thought to undercut the idea that visual experience is informationally rich (inattentional blindness, etc.), along with theories of vision according to which "ensemble statistics" are used to represent features in the periphery of the visual field. I'll argue that these lines of evidence fail to close the apparently vast gap between the amount of visual content represented in the visual experience and the amount represented in the PFC.

Taking consciousness for real: Increasing the ecological validity of the study of conscious vs. unconscious processes

The study of consciousness has developed well-controlled, rigorous methods for manipulating and measuring consciousness. Yet, in the process, experimental paradigms grew farther away from everyday conscious and unconscious processes, which raises the concern of ecological validity. In this review, we suggest that the field can benefit from adopting a more ecological approach, akin to other fields of cognitive science. There, this approach challenged some existing hypotheses, yielded stronger effects, and enabled new research questions. We argue that such a move is critical for studying consciousness, where experimental paradigms tend to be artificial and small effect sizes are relatively prevalent. We identify three paths for doing so-changing the stimuli and experimental settings, changing the measures, and changing the research questions themselves-and review works that have already started implementing such approaches. While acknowledging the inherent challenges, we call for increasing ecological validity in consciousness studies.

(Dis)confirming theories of consciousness and their predictions: towards a Lakatosian consciousness science

The neuroscience of consciousness is undergoing a significant empirical acceleration thanks to several adversarial collaborations that intend to test different predictions of rival theories of consciousness. In this context, it is important to pair consciousness science with confirmation theory, the philosophical discipline that explores the interaction between evidence and hypotheses, in order to understand how exactly, and to what extent, specific experiments are challenging or validating theories of consciousness. In this paper, I examine this intricate relationship by adopting a Lakatosian lens. I propose that Lakatos' philosophy of science can aid consciousness scientists to better interpret adversarial collaborations in consciousness science and, more generally, to develop a confirmation-theoretic model of theory-appraisal in this field. I do so by suggesting that such a model be built upon three Lakatos-inspired criteria for assessing the relationship between empirical evidence and theoretical predictions: (i) the model should represent the 'distinction between prediction and accommodation'; (ii) the model should represent the 'structural relevance' of predictions; (iii) the model should represent the 'boldness' of the predictions. I argue that a Lakatosian model of theory-appraisal has both normative and descriptive virtues, and can move the debate forward by acknowledging that theory-appraisal needs to consider the diachronic development of theories, their logical structure, and their relationship with background beliefs and knowledge.

An adjunction hypothesis between qualia and reports

What are the nature of the relationship among qualia, contents of consciousness, and behavioral reports? Traditionally, this type of question has been only addressed via a qualitative and philosophical approach. Some theorists emphasize an incomplete and inaccurate nature of reports of one's own qualia to discourage formal research programs on qualia. Other empirical researchers, however, have made substantial progress in understanding the structure of qualia from such limited reports. What is the precise relationship between the two? To answer this question, we introduce the concept of “adjoint” or “adjunction” from the category theory in mathematics. We claim that the adjunction captures some aspects of the nuanced relationships between qualia and reports. The concept of adjunction allows us to clarify the conceptual issues with a precise mathematical formulation. In particular, adjunction establishes coherence between two categories that cannot be considered equivalent, yet has an important relationship. This rises in empirical experimental situations between qualia and reports. More importantly, an idea of adjunction naturally leads to various proposals of new empirical experiments to test the predictions about the nature of their relationship as well as other issues in consciousness research.

How much consciousness is there in complexity?

The notion of complexity currently receives significant attention in neuroscience, mainly through the popularity of the Integrated Information Theory (IIT). It has proven successful in research centred on discriminating states of consciousness, while little theoretical and experimental effort was directed toward studying the content. In this paper, we argue that exploring the relationship between complexity and conscious content is necessary to understand the importance of information-theoretic measures for consciousness research properly. We outline how content could be experimentally operationalised and how rudimental testable hypotheses can be formulated without requiring IIT formalisms. This approach would not only allow for a better understanding of aspects of consciousness captured by complexity but could also facilitate comparison efforts for theories of consciousness.

Perception of the non-dominant hand as larger after non-judgmental focus on its details

We investigated whether brief non-judgmental focus on the details of one's non-dominant hand might lead to changes in perception of its size, and if such a change would be related to central coherence, body dissatisfaction, or how much participants liked their hand. After two pilot experiments (N = 28 and N = 30 respectively: Appendix 1), a within-subject experiment (N = 82) was conducted. Subjects were mainly university students. They were asked to rate the size of their non-dominant hand and how much they liked it, and the size of an external object (a X-box controller) on a visual-analog scale before and after focusing on their details for 5 min, as well as the size of another object (a calculator) before and after a 5 min long distraction task. After completing the tasks, they were asked to respond to a brief questionnaire on body dissatisfaction. A s significant interaction between time and factors (non-dominant hand, X-box controller and calculator) emerged (F(2, 78) = 6.41, p = .003). Participants rated their hand as larger after focusing on its details compared to baseline, and this change was significantly larger than those reported for the X-box controller. No significant change in how they liked their hand was observed, and contrary to the pilot experiments, the perceived change in size of the hand was not related to body dissatisfaction. The significant change in reporting of the size of the hand after focusing on its details seems to be an interesting finding, worth further replications.

How much can we differentiate at a brief glance: revealing the truer limit in conscious contents through the massive report paradigm (MRP)

Upon a brief glance, how well can we differentiate what we see from what we do not? Previous studies answered this question as 'poorly'. This is in stark contrast with our everyday experience. Here, we consider the possibility that previous restriction in stimulus variability and response alternatives reduced what participants could express from what they consciously experienced. We introduce a novel massive report paradigm that probes the ability to differentiate what we see from what we do not. In each trial, participants viewed a natural scene image and judged whether a small image patch was a part of the original image. To examine the limit of discriminability, we also included subtler changes in the image as modification of objects. Neither the images nor patches were repeated per participant. Our results showed that participants were highly accurate (accuracy greater than 80%) in differentiating patches from the viewed images from patches that are not present. Additionally, the differentiation between original and modified objects was influenced by object sizes and/or the congruence between objects and the scene gists. Our massive report paradigm opens a door to quantitatively measure the limit of immense informativeness of a moment of consciousness.

Consciousness matters: phenomenal experience has functional value

‘Why would we do anything at all if the doing was not doing something to us?’ In other words: What is consciousness good for? Here, reversing classical views, according to many of which subjective experience is a mere epiphenomenon that affords no functional advantage, we propose that subject-level experience—‘What it feels like’—is endowed with intrinsic value, and it is precisely the value agents associate with their experiences that explains why they do certain things and avoid others. Because experiences have value and guide behaviour, consciousness has a function. Under this hypothesis of ‘phenomenal worthiness’, we argue that it is only in virtue of the fact that conscious agents ‘experience’ things and ‘care’ about those experiences that they are ‘motivated’ to act in certain ways and that they ‘prefer’ some states of affairs vs. others. Overviewing how the concept of value has been approached in decision-making, emotion research and consciousness research, we argue that phenomenal consciousness has intrinsic value and conclude that if this is indeed the case, then it must have a function. Phenomenal experience might act as a mental currency of sorts, which not only endows conscious mental states with intrinsic value but also makes it possible for conscious agents to compare vastly different experiences in a common subject-centred space—a feature that readily explains the fact that consciousness is ‘unified’. The phenomenal worthiness hypothesis, in turn, makes the ‘hard problem’ of consciousness more tractable, since it can then be reduced to a problem about function.

Finding meaning in "wrong responses": The multiple object-awareness paradigm shows that visual awareness is probabilistic

Visual information that observers perceive and remember at any given moment guides behavior in daily life. However, binary alternative-forced choice responses, often used in visual research, limit the report of the visual information that observers perceive and remember. We used a new multiple object-awareness paradigm where observers can use multiple clicks to find a target. We calculated visual awareness capacity based on the first-attempt accuracy and the total number of clicks, respectively. Results showed that the capacity estimated by the clicks in guessing from N was significantly greater than that estimated by the first-attempt accuracy. Further, analysis found that if observers could not locate the target in their first attempt, they were more likely to click closer to the target or on stimuli that matched its color. In addition, we found that even when observers used the same number of clicks to find a target (2 or 3), the average distance was shorter when observers reported high-level subjective visibility. The findings are compatible with the partial awareness hypothesis, and the visual ensembles and summary statistics hypothesis, which hold that visual awareness is probabilistic. These results also support the visual short-term memory models where many items are stored but with a resolution or noise level that depends on the number of items in memory.

What can we experience and report on a rapidly presented image? Intersubjective measures of specificity of freely reported contents of consciousness

Background: A majority of previous studies appear to support a view that human observers can only perceive coarse information from a natural scene image when it is presented rapidly (<100ms, masked). In these studies, participants were often forced to choose an answer from options that experimenters preselected. These options can underestimate what participants experience and can report on it. The current study aims to introduce a novel methodology to investigate how detailed information participants can report after briefly seeing a natural scene image.     Methods: We used a novel free-report paradigm to examine what people can freely report following a rapidly presented natural scene image (67/133/267ms, masked). N = 600 online participants typed up to five words to report what they saw in the image together with confidence of the respective responses. We developed a novel index, Intersubjective Agreement (IA). IA quantifies how specifically the response words were used to describe the target image, with a high value meaning the word is not often reported for other images. Importantly, IA eliminates the need for experimenters to preselect response options. Results: The words with high IA values are often something detailed (e.g., a small object) in a particular image. With IA, unlike commonly believed, we demonstrated that participants reported highly specific and detailed aspects of the briefly (even at 67ms, masked) shown image. Further, IA is positively correlated with confidence, indicating metacognitive conscious access to the reported aspects of the image. Conclusion: These new findings challenge the dominant view that the content of rapid scene experience is limited to global and coarse gist. Our novel paradigm opens a door to investigate various contents of consciousness with a free-report paradigm.

What can we experience and report on a rapidly presented image? Intersubjective measures of specificity of freely reported contents of consciousness

Background: A majority of previous studies appear to support a view that human observers can only perceive coarse information from a natural scene image when it is presented rapidly (<100ms, masked). In these studies, participants were often forced to choose an answer from options that experimenters preselected. These options can underestimate what participants experience and can report on it. The current study aims to introduce a novel methodology to investigate how detailed information participants can report after briefly seeing a natural scene image.     Methods: We used a novel free-report paradigm to examine what people can freely report following a rapidly presented natural scene image (67/133/267ms, masked). N = 670 online participants typed up to five words to report what they saw in the image together with confidence of the respective responses. We developed a novel index, Intersubjective Agreement (IA). IA quantifies how specifically the response words were used to describe the target image, with a high value meaning the word is not often reported for other images. Importantly, IA eliminates the need for experimenters to preselect response options. Results: The words with high IA values are often something detailed (e.g., a small object) in a particular image. With IA, unlike commonly believed, we demonstrated that participants reported highly specific and detailed aspects of the briefly (even at 67ms, masked) shown image. Further, IA is positively correlated with confidence, indicating metacognitive conscious access to the reported aspects of the image. Conclusion: These new findings challenge the dominant view that the content of rapid scene experience is limited to global and coarse gist. Our novel paradigm opens a door to investigate various contents of consciousness with a free-report paradigm.

What can we experience and report on a rapidly presented image? Intersubjective measures of specificity of freely reported contents of consciousness

Background: A majority of previous studies appear to support a view that human observers can only perceive coarse information from a natural scene image when it is presented rapidly (<100ms, masked). In these studies, participants were often forced to choose an answer from options that experimenters preselected. These options can underestimate what participants experience and can report on it. The current study aims to introduce a novel methodology to investigate how detailed information participants can report after briefly seeing a natural scene image. Methods: We used a novel free-report paradigm to examine what people can freely report following a rapidly presented natural scene image (67/133/267ms, masked). N = 670 online participants typed up to five words to report what they saw in the image together with confidence of the respective responses. We developed a novel index, Intersubjective Agreement (IA). IA quantifies how specifically the response words were used to describe the target image, with a high value meaning the word is not often reported for other images. Importantly, IA eliminates the need for experimenters to preselect response options. Results: The words with high IA values are often something detailed (e.g., a small object) in a particular image. With IA, unlike commonly believed, we demonstrated that participants reported highly specific and detailed aspects of the briefly (even at 67ms, masked) shown image. Further, IA is positively correlated with confidence, indicating metacognitive conscious access to the reported aspects of the image. Conclusion: These new findings challenge the dominant view that the content of rapid scene experience is limited to global and coarse gist. Our novel paradigm opens a door to investigate various contents of consciousness with a free-report paradigm.

A relational approach to consciousness: categories of level and contents of consciousness

Characterizing consciousness in and of itself is notoriously difficult. Here, we propose an alternative approach to characterize, and eventually define, consciousness through exhaustive descriptions of consciousness' relationships to all other consciousness. This approach is founded in category theory. Indeed, category theory can prove that two objects A and B in a category can be equivalent if and only if all the relationships that A holds with others in the category are the same as those of B; this proof is called the Yoneda lemma. To introduce the Yoneda lemma, we gradually introduce key concepts of category theory to consciousness researchers. Along the way, we propose several possible definitions of categories of consciousness, both in terms of level and contents, through the usage of simple examples. We propose to use the categorical structure of consciousness as a gold standard to formalize empirical research (e.g. color qualia structure at fovea and periphery) and, especially, the empirical testing of theories of consciousness.

Lang J, M. Haun A, Albantakis L, Boly M, Tononi G. Consciousness and the fallacy of misplaced objectivity

Objective correlates-behavioral, functional, and neural-provide essential tools for the scientific study of consciousness. But reliance on these correlates should not lead to the 'fallacy of misplaced objectivity': the assumption that only objective properties should and can be accounted for objectively through science. Instead, what needs to be explained scientifically is what experience is intrinsically-its subjective properties-not just what we can do with it extrinsically. And it must be explained; otherwise the way experience feels would turn out to be magical rather than physical. We argue that it is possible to account for subjective properties objectively once we move beyond cognitive functions and realize what experience is and how it is structured. Drawing on integrated information theory, we show how an objective science of the subjective can account, in strictly physical terms, for both the essential properties of every experience and the specific properties that make particular experiences feel the way they do.

Measures of explicit and implicit in motor learning: what we know and what we don't

Adaptation tasks are a key tool in characterizing the contribution of explicit and implicit processes to sensorimotor learning. However, different assumptions and ideas underlie methods used to measure these processes, leading to inconsistencies between studies. For instance, it is still unclear explicit and implicit combine additively. Cognitive studies of explicit and implicit processes show how non-additivity and bias in measurement can distort results. We argue that to understand explicit and implicit processes in visuomotor adaptation, we need a stronger characterization of the phenomenology and a richer set of models to test it on.

What is visible across the visual field?

It is sometimes claimed that because the resolution and sensitivity of visual perception are better in the fovea than in the periphery, peripheral vision cannot support the same kinds of colour and sharpness percepts as foveal vision. The fact that a scene nevertheless seems colourful and sharp throughout the visual field then poses a puzzle. In this study, I use a detailed model of human spatial vision to estimate the visibility of certain properties of natural scenes, including aspects of colourfulness, sharpness, and blurriness, across the visual field. The model is constructed to reproduce basic aspects of human contrast and colour sensitivity over a range of retinal eccentricities. I apply the model to colourful, complex natural scene images, and estimate the degree to which colour and edge information are present in the model's representation of the scenes. I find that, aside from the intrinsic drift in the spatial scale of the representation, there are not large qualitative differences between foveal and peripheral representations of 'colourfulness' and 'sharpness'.

A review of interactions between peripheral and foveal vision

Visual processing varies dramatically across the visual field. These differences start in the retina and continue all the way to the visual cortex. Despite these differences in processing, the perceptual experience of humans is remarkably stable and continuous across the visual field. Research in the last decade has shown that processing in peripheral and foveal vision is not independent, but is more directly connected than previously thought. We address three core questions on how peripheral and foveal vision interact, and review recent findings on potentially related phenomena that could provide answers to these questions. First, how is the processing of peripheral and foveal signals related during fixation? Peripheral signals seem to be processed in foveal retinotopic areas to facilitate peripheral object recognition, and foveal information seems to be extrapolated toward the periphery to generate a homogeneous representation of the environment. Second, how are peripheral and foveal signals re-calibrated? Transsaccadic changes in object features lead to a reduction in the discrepancy between peripheral and foveal appearance. Third, how is peripheral and foveal information stitched together across saccades? Peripheral and foveal signals are integrated across saccadic eye movements to average percepts and to reduce uncertainty. Together, these findings illustrate that peripheral and foveal processing are closely connected, mastering the compromise between a large peripheral visual field and high resolution at the fovea.

Hallucinations on demand: the utility of experimentally induced phenomena in hallucination research

Despite the desire to delve deeper into hallucinations of all types, methodological obstacles have frustrated development of more rigorous quantitative experimental techniques, thereby hampering research progress. Here, we discuss these obstacles and, with reference to visual phenomena, argue that experimentally induced phenomena (e.g. hallucinations induced by flickering light and classical conditioning) can bring hallucinations within reach of more objective behavioural and neural measurement. Expanding the scope of hallucination research raises questions about which phenomena qualify as hallucinations, and how to identify phenomena suitable for use as laboratory models of hallucination. Due to the ambiguity inherent in current hallucination definitions, we suggest that the utility of phenomena for use as laboratory hallucination models should be represented on a continuous spectrum, where suitability varies with the degree to which external sensory information constrains conscious experience. We suggest that existing strategies that group pathological hallucinations into meaningful subtypes based on hallucination characteristics (including phenomenology, disorder and neural activity) can guide extrapolation from hallucination models to other hallucinatory phenomena. Using a spectrum of phenomena to guide scientific hallucination research should help unite the historically separate fields of psychophysics, cognitive neuroscience and clinical research to better understand and treat hallucinations, and inform models of consciousness. This article is part of the theme issue 'Offline perception: voluntary and spontaneous perceptual experiences without matching external stimulation'.

Defending subjective inflation: an inference to the best explanation

In a recent opinion piece, Abid (2019) criticizes the hypothesis that subjective inflation may partly account for apparent phenomenological richness across the visual field and outside the focus of attention. In response, we address three main issues. First, we maintain that inflation should be interpreted as an intraperceptual-and not post-perceptual-phenomenon. Second, we describe how inflation may differ from filling-in. Finally, we contend that, in general, there is sufficient evidence to tip the scales toward intraperceptual interpretations of visibility and confidence judgments.

Visual experience forms a multidimensional pattern that is not reducible to a single measure: Evidence from metacontrast masking

A metacontrast masking paradigm was employed to provide evidence for the richness and diversity of our visual experience. Square- and diamond-shaped targets were followed by square- and diamond-shaped masks at varying stimulus onset asynchronies (SOAs), resulting in shape-congruent and shape-incongruent trials. In Experiment 1, participants reported in each trial how they perceived target and mask. After extended training, seven different aspects of the target could be distinguished as specific percepts in this metacontrast masking paradigm. These percepts encompass aspects including the temporal distance between both stimuli, the perceived contrast of the target, and motion percepts resulting from the interplay between the target and mask. Participants spontaneously reported each of these percepts, and the frequency of reports varied systematically with SOA and the congruency between target and mask. In Experiment 2, we trained a new group of participants to distinguish each of these target percepts. Again, the frequency of reports of the specific percepts varied with SOA and congruency, just as in Experiment 1. In a last session, we measured objective discrimination performance yielding the typical individually different masking functions across SOAs. An examination of the relation between the frequencies of reports of subjective percepts and objective discrimination performance revealed multiple dissociations between these measures. Results suggest a multidimensional pattern of subjective experiences under metacontrast, which is reflected in dissociated subjective and objective measures of visual awareness. As a consequence, awareness cannot be assessed exhaustively by a single measure, thus challenging the use of simple one-dimensional subjective or objective measures in visual masking.

The limits of color awareness during active, real-world vision

Color ignites visual experience, imbuing the world with meaning, emotion, and richness. As soon as an observer opens their eyes, they have the immediate impression of a rich, colorful experience that encompasses their entire visual world. Here, we show that this impression is surprisingly inaccurate. We used head-mounted virtual reality (VR) to place observers in immersive, dynamic real-world environments, which they naturally explored via saccades and head turns. Meanwhile, we monitored their gaze with in-headset eye tracking and then systematically altered the visual environments such that only the parts of the scene they were looking at were presented in color and the rest of the scene (i.e., the visual periphery) was entirely desaturated. We found that observers were often completely unaware of these drastic alterations to their visual world. In the most extreme case, almost a third of observers failed to notice when less than 5% of the visual display was presented in color. This limitation on perceptual awareness could not be explained by retinal neuroanatomy or previous studies of peripheral visual processing using more traditional psychophysical approaches. In a second study, we measured color detection thresholds using a staircase procedure while a set of observers intentionally attended to the periphery. Still, we found that observers were unaware when a large portion of their field of view was desaturated. Together, these results show that during active, naturalistic viewing conditions, our intuitive sense of a rich, colorful visual world is largely incorrect.

Demystifying visual awareness: Peripheral encoding plus limited decision complexity resolve the paradox of rich visual experience and curious perceptual failures

Human beings subjectively experience a rich visual percept. However, when behavioral experiments probe the details of that percept, observers perform poorly, suggesting that vision is impoverished. What can explain this awareness puzzle? Is the rich percept a mere illusion? How does vision work as well as it does? This paper argues for two important pieces of the solution. First, peripheral vision encodes its inputs using a scheme that preserves a great deal of useful information, while losing the information necessary to perform certain tasks. The tasks rendered difficult by the peripheral encoding include many of those used to probe the details of visual experience. Second, many tasks used to probe attentional and working memory limits are, arguably, inherently difficult, and poor performance on these tasks may indicate limits on decision complexity. Two assumptions are critical to making sense of this hypothesis: (1) All visual perception, conscious or not, results from performing some visual task; and (2) all visual tasks face the same limit on decision complexity. Together, peripheral encoding plus decision complexity can explain a wide variety of phenomena, including vision's marvelous successes, its quirky failures, and our rich subjective impression of the visual world.

Visual experience forms a multidimensional pattern that is not reducible to a single measure: Evidence from metacontrast masking

A metacontrast masking paradigm was employed to provide evidence for the richness and diversity of our visual experience. Square- and diamond-shaped targets were followed by square- and diamond-shaped masks at varying stimulus onset asynchronies (SOAs), resulting in shape-congruent and shape-incongruent trials. In Experiment 1, participants reported in each trial how they perceived target and mask. After extended training, seven different aspects of the target could be distinguished as specific percepts in this metacontrast masking paradigm. These percepts encompass aspects including the temporal distance between both stimuli, the perceived contrast of the target, and motion percepts resulting from the interplay between the target and mask. Participants spontaneously reported each of these percepts, and the frequency of reports varied systematically with SOA and the congruency between target and mask. In Experiment 2, we trained a new group of participants to distinguish each of these target percepts. Again, the frequency of reports of the specific percepts varied with SOA and congruency, just as in Experiment 1. In a last session, we measured objective discrimination performance yielding the typical individually different masking functions across SOAs. An examination of the relation between the frequencies of reports of subjective percepts and objective discrimination performance revealed multiple dissociations between these measures. Results suggest a multidimensional pattern of subjective experiences under metacontrast, which is reflected in dissociated subjective and objective measures of visual awareness. As a consequence, awareness cannot be assessed exhaustively by a single measure, thus challenging the use of simple one-dimensional subjective or objective measures in visual masking.

Why Does Space Feel the Way it Does? Towards a Principled Account of Spatial Experience

There must be a reason why an experience feels the way it does. A good place to begin addressing this question is spatial experience, because it may be more penetrable by introspection than other qualities of consciousness such as color or pain. Moreover, much of experience is spatial, from that of our body to the visual world, which appears as if painted on an extended canvas in front of our eyes. Because it is ‘right there’, we usually take space for granted and overlook its qualitative properties. However, we should realize that a great number of phenomenal distinctions and relations are required for the canvas of space to feel ‘extended’. Here we argue that, to be experienced as extended, the canvas of space must be composed of countless spots, here and there, small and large, and these spots must be related to each other in a characteristic manner through connection, fusion, and inclusion. Other aspects of the structure of spatial experience follow from extendedness: every spot can be experienced as enclosing a particular region, with its particular location, size, boundary, and distance from other spots. We then propose an account of the phenomenal properties of spatial experiences based on integrated information theory (IIT). The theory provides a principled approach for characterizing both the quantity and quality of experience by unfolding the cause-effect structure of a physical substrate. Specifically, we show that a simple simulated substrate of units connected in a grid-like manner yields a cause-effect structure whose properties can account for the main properties of spatial experience. These results uphold the hypothesis that our experience of space is supported by brain areas whose units are linked by a grid-like connectivity. They also predict that changes in connectivity, even in the absence of changes in activity, should lead to a warping of experienced space. To the extent that this approach provides an initial account of phenomenal space, it may also serve as a starting point for investigating other aspects of the quality of experience and their physical correspondents.

Deflating inflation: the connection (or lack thereof) between decisional and metacognitive processes and visual phenomenology

Vision presents us with a richly detailed world. Yet, there is a range of limitations in the processing of visual information, such as poor peripheral resolution and failures to notice things we do not attend. This raises a natural question: How do we seem to see so much when there is considerable evidence indicating otherwise? In an elegant series of studies, Lau and colleagues have offered a novel answer to this long-standing question, proposing that our sense of visual richness is an artifact of decisional and metacognitive deficits. I critically evaluate this proposal and conclude that it rests on questionable presuppositions concerning the relationship between decisional and metacognitive processes, on one hand, and visual phenomenology, on the other.

Causal Composition: Structural Differences among Dynamically Equivalent Systems

The dynamical evolution of a system of interacting elements can be predicted in terms of its elementary constituents and their interactions, or in terms of the system’s global state transitions. For this reason, systems with equivalent global dynamics are often taken to be equivalent for all relevant purposes. Nevertheless, such systems may still vary in their causal composition—the way mechanisms within the system specify causes and effects over different subsets of system elements. We demonstrate this point based on a set of small discrete dynamical systems with reversible dynamics that cycle through all their possible states. Our analysis elucidates the role of composition within the formal framework of integrated information theory. We show that the global dynamical and information-theoretic capacities of reversible systems can be maximal even though they may differ, quantitatively and qualitatively, in the information that their various subsets specify about each other (intrinsic information). This can be the case even for a system and its time-reversed equivalent. Due to differences in their causal composition, two systems with equivalent global dynamics may still differ in their capacity for autonomy, agency, and phenomenology.

The methodological puzzle of phenomenal consciousness

Is phenomenal consciousness constitutively related to cognitive access? Despite being a fundamental issue for any science of consciousness, its empirical study faces a severe methodological puzzle. Recent years have seen numerous attempts to address this puzzle, either in practice, by offering evidence for a positive or negative answer, or in principle, by proposing a framework for eventual resolution. The present paper critically considers these endeavours, including partial-report, metacognitive and no-report paradigms, as well as the theoretical proposal that we can make progress by studying phenomenal consciousness as a natural kind. It is argued that the methodological puzzle remains obdurately with us and that, for now, we must adopt an attitude of humility towards the phenomenal.This article is part of the theme issue 'Perceptual consciousness and cognitive access'.

If perception is probabilistic, why does it not seem probabilistic?

The success of the Bayesian perspective in explaining perceptual phenomena has motivated the view that perceptual representation is probabilistic. But if perceptual representation is probabilistic, why does normal conscious perception not reflect the full probability functions that the probabilistic point of view endorses? For example, neurons in cortical area MT that respond to the direction of motion are broadly tuned: a patch of cortex that is tuned to vertical motion also responds to horizontal motion, but when we see vertical motion, foveally, in good conditions, it does not look at all horizontal. The standard solution in terms of sampling runs into the problem that sampling is an account of perceptual decision rather than perception. This paper argues that the best Bayesian approach to this problem does not require probabilistic representation.This article is part of the theme issue 'Perceptual consciousness and cognitive access'.

Inflation versus filling-in: why we feel we see more than we actually do in peripheral vision

Do we perceive fine details in the visual periphery? Here, we propose that phenomenology in the visual periphery can be characterized by an inflated sense of perceptual capacity, as observers overestimate the quality of their perceptual inputs. Distinct from the well-known perceptual phenomenon of 'filling-in' where perceptual content is generated or completed endogenously, inflation can be characterized by incorrect introspection at the subjective level. The perceptual content itself may be absent or weak (i.e. not necessarily filled-in), and yet such content is mistakenly regarded by the system as rich. Behaviourally, this can be reflected by metacognitive deficits in the degree to which confidence judgements track task accuracy, and decisional biases for observers to think particular items are present, even when they are not. In two experiments using paradigms that exploit unique attributes of peripheral vision (crowding and summary statistics), we provide evidence that both types of deficits are present in peripheral vision, as observers' reports are marked by overconfidence in discrimination judgements and high numbers of false alarms in detection judgements. We discuss potential mechanisms that may be the cause of inflation and propose future experiments to further explore this unique sensory phenomenon.This article is part of the theme issue 'Perceptual consciousness and cognitive access'.

Image content is more important than Bouma's Law for scene metamers

We subjectively perceive our visual field with high fidelity, yet peripheral distortions can go unnoticed and peripheral objects can be difficult to identify (crowding). Prior work showed that humans could not discriminate images synthesised to match the responses of a mid-level ventral visual stream model when information was averaged in receptive fields with a scaling of about half their retinal eccentricity. This result implicated ventral visual area V2, approximated ‘Bouma’s Law’ of crowding, and has subsequently been interpreted as a link between crowding zones, receptive field scaling, and our perceptual experience. However, this experiment never assessed natural images. We find that humans can easily discriminate real and model-generated images at V2 scaling, requiring scales at least as small as V1 receptive fields to generate metamers. We speculate that explaining why scenes look as they do may require incorporating segmentation and global organisational constraints in addition to local pooling.

As you read this digest, your eyes move to follow the lines of text. But now try to hold your eyes in one position, while reading the text on either side and below: it soon becomes clear that peripheral vision is not as good as we tend to assume. It is not possible to read text far away from the center of your line of vision, but you can see ‘something’ out of the corner of your eye. You can see that there is text there, even if you cannot read it, and you can see where your screen or page ends. So how does the brain generate peripheral vision, and why does it differ from what you see when you look straight ahead?

One idea is that the visual system averages information over areas of the peripheral visual field. This gives rise to texture-like patterns, as opposed to images made up of fine details. Imagine looking at an expanse of foliage, gravel or fur, for example. Your eyes cannot make out the individual leaves, pebbles or hairs. Instead, you perceive an overall pattern in the form of a texture. Our peripheral vision may also consist of such textures, created when the brain averages information over areas of space.

Wallis, Funke et al. have now tested this idea using an existing computer model that averages visual input in this way. By giving the model a series of photographs to process, Wallis, Funke et al. obtained images that should in theory simulate peripheral vision. If the model mimics the mechanisms that generate peripheral vision, then healthy volunteers should be unable to distinguish the processed images from the original photographs. But in fact, the participants could easily discriminate the two sets of images. This suggests that the visual system does not solely use textures to represent information in the peripheral visual field. Wallis, Funke et al. propose that other factors, such as how the visual system separates and groups objects, may instead determine what we see in our peripheral vision.

This knowledge could ultimately benefit patients with eye diseases such as macular degeneration, a condition that causes loss of vision in the center of the visual field and forces patients to rely on their peripheral vision.

What Is the True Capacity of Visual Cognition?

How much can we perceive and remember at a time? Results from various paradigms traditionally show that observers are aware of surprisingly little of the world around them. However, a recent study by Wu and Wolfe (Curr. Biol. 2018;28:3430-3434) uses a novel technique to reveal that observers have more knowledge of the visual world than previously believed.

The relationship between attention and consciousness: an expanded taxonomy and implications for 'no-report' paradigms

Tensions between global neuronal workspace theory and recurrent processing theory have sparked much debate in the field of consciousness research. Here, we focus on one of the key distinctions between these theories: the proposed relationship between attention and consciousness. By reviewing recent empirical evidence, we argue that both theories contain key insights and that certain aspects of each theory can be reconciled into a novel framework that may help guide future research. Alternative theories are also considered, including attended intermediate-level representations theory, integrated information theory and higher order thought theory. With the aim of offering a fresh and nuanced perspective to current theoretical debates, an updated taxonomy of conscious and non-conscious states is proposed. This framework maps a wider spectrum of conscious states by incorporating contemporary views from cognitive neuroscience regarding the variety of attentional mechanisms that are known to interact with sensory processing. Whether certain types of attention are necessary for phenomenal and access consciousness is considered and incorporated into this extended taxonomy. To navigate this expanded space, we review recent 'no-report' paradigms and address several methodological misunderstandings in order to pave a clear path forward for identifying the neural basis of perceptual awareness.This article is part of the theme issue 'Perceptual consciousness and cognitive access'.

The Illusion of Uniformity Does Not Depend on the Primary Visual Cortex: Evidence From Sensory Adaptation

Visual experience appears richly detailed despite the poor resolution of the majority of the visual field, thanks to foveal-peripheral integration. The recently described uniformity illusion (UI), wherein peripheral elements of a pattern take on the appearance of foveal elements, may shed light on this integration. We examined the basis of UI by generating adaptation to a pattern of Gabors suitable for producing UI on orientation. After removing the pattern, participants reported the tilt of a single peripheral Gabor. The tilt aftereffect followed the physical adapting orientation rather than the global orientation perceived under UI, even when the illusion had been reported for a long time. Conversely, a control experiment replacing illusory uniformity with a physically uniform Gabor pattern for the same durations did produce an aftereffect to the global orientation. Results indicate that UI is not associated with changes in sensory encoding at V1 but likely depends on higher level processes.