Recognizing sights, smells, and sounds with gnostic fields

How Convolutional Neural Network Architecture Biases Learned Opponency and Color Tuning

Recent work suggests that changing convolutional neural network (CNN) architecture by introducing a bottleneck in the second layer can yield changes in learned function. To understand this relationship fully requires a way of quantitatively comparing trained networks. The fields of electrophysiology and psychophysics have developed a wealth of methods for characterizing visual systems that permit such comparisons. Inspired by these methods, we propose an approach to obtaining spatial and color tuning curves for convolutional neurons that can be used to classify cells in terms of their spatial and color opponency. We perform these classifications for a range of CNNs with different depths and bottleneck widths. Our key finding is that networks with a bottleneck show a strong functional organization: almost all cells in the bottleneck layer become both spatially and color opponent, and cells in the layer following the bottleneck become nonopponent. The color tuning data can further be used to form a rich understanding of how color a network encodes color. As a concrete demonstration, we show that shallower networks without a bottleneck learn a complex nonlinear color system, whereas deeper networks with tight bottlenecks learn a simple channel opponent code in the bottleneck layer. We develop a method of obtaining a hue sensitivity curve for a trained CNN that enables high-level insights that complement the low-level findings from the color tuning data. We go on to train a series of networks under different conditions to ascertain the robustness of the discussed results. Ultimately our methods and findings coalesce with prior art, strengthening our ability to interpret trained CNNs and furthering our understanding of the connection between architecture and learned representation. Trained models and code for all experiments are available at https://github.com/ecs-vlc/opponency.

Machine Learning in Human Olfactory Research

The complexity of the human sense of smell is increasingly reflected in complex and high-dimensional data, which opens opportunities for data-driven approaches that complement hypothesis-driven research. Contemporary developments in computational and data science, with its currently most popular implementation as machine learning, facilitate complex data-driven research approaches. The use of machine learning in human olfactory research included major approaches comprising 1) the study of the physiology of pattern-based odor detection and recognition processes, 2) pattern recognition in olfactory phenotypes, 3) the development of complex disease biomarkers including olfactory features, 4) odor prediction from physico-chemical properties of volatile molecules, and 5) knowledge discovery in publicly available big databases. A limited set of unsupervised and supervised machine-learned methods has been used in these projects, however, the increasing use of contemporary methods of computational science is reflected in a growing number of reports employing machine learning for human olfactory research. This review provides key concepts of machine learning and summarizes current applications on human olfactory data.

Object concepts in the chemical senses

This paper examines the applicability of the object concept to the chemical senses, by evaluating them against a set of criteria for object-hood. Taste and chemesthesis do not generate objects. Their parts, perceptible from birth, never combine. Orthonasal olfaction (sniffing) presents a strong case for generating objects. Odorants have many parts yet they are perceived as wholes, this process is based on learning, and there is figure-ground segregation. While flavors are multimodal representations bound together by learning, there is no functional need for flavor objects in the mouth. Rather, food identification occurs prior to ingestion using the eye and nose, with the latter retrieving multimodal flavor objects via sniffing (e.g., sweet smelling caramel). While there are differences in object perception between vision, audition, and orthonasal olfaction, the commonalities suggest that the brain has adopted the same basic solution when faced with extracting meaning from complex stimulus arrays.