Neural Projection Mapping Using Reflectance Fields

We introduce a high resolution spatially adaptive light source, or a projector, into a neural reflectance field that allows to both calibrate the projector and photo realistic light editing. The projected texture is fully differentiable with respect to all scene parameters, and can be optimized to yield a desired appearance suitable for applications in augmented reality and projection mapping. Our neural field consists of three neural networks, estimating geometry, material, and transmittance. Using an analytical BRDF model and carefully selected projection patterns, our acquisition process is simple and intuitive, featuring a fixed uncalibrated projected and a handheld camera with a co-located light source. As we demonstrate, the virtual projector incorporated into the pipeline improves scene understanding and enables various projection mapping applications, alleviating the need for time consuming calibration steps performed in a traditional setting per view or projector location. In addition to enabling novel viewpoint synthesis, we demonstrate state-of-the-art performance projector compensation for novel viewpoints, improvement over the baselines in material and scene reconstruction, and three simply implemented scenarios where projection image optimization is performed, including the use of a 2D generative model to consistently dictate scene appearance from multiple viewpoints. We believe that neural projection mapping opens up the door to novel and exciting downstream tasks, through the joint optimization of the scene and projection images.

iCatcher+: Robust and Automated Annotation of Infants' and Young Children's Gaze Behavior From Videos Collected in Laboratory, Field, and Online Studies

Technological advances in psychological research have enabled large-scale studies of human behavior and streamlined pipelines for automatic processing of data. However, studies of infants and children have not fully reaped these benefits because the behaviors of interest, such as gaze duration and direction, still have to be extracted from video through a laborious process of manual annotation, even when these data are collected online. Recent advances in computer vision raise the possibility of automated annotation of these video data. In this article, we built on a system for automatic gaze annotation in young children, iCatcher, by engineering improvements and then training and testing the system (referred to hereafter as iCatcher+) on three data sets with substantial video and participant variability (214 videos collected in U.S. lab and field sites, 143 videos collected in Senegal field sites, and 265 videos collected via webcams in homes; participant age range = 4 months-3.5 years). When trained on each of these data sets, iCatcher+ performed with near human-level accuracy on held-out videos on distinguishing "LEFT" versus "RIGHT" and "ON" versus "OFF" looking behavior across all data sets. This high performance was achieved at the level of individual frames, experimental trials, and study videos; held across participant demographics (e.g., age, race/ethnicity), participant behavior (e.g., movement, head position), and video characteristics (e.g., luminance); and generalized to a fourth, entirely held-out online data set. We close by discussing next steps required to fully automate the life cycle of online infant and child behavioral studies, representing a key step toward enabling robust and high-throughput developmental research.

iCatcher: A neural network approach for automated coding of young children's eye movements

Infants' looking behaviors are often used for measuring attention, real-time processing, and learning-often using low-resolution videos. Despite the ubiquity of gaze-related methods in developmental science, current analysis techniques usually involve laborious post hoc coding, imprecise real-time coding, or expensive eye trackers that may increase data loss and require a calibration phase. As an alternative, we propose using computer vision methods to perform automatic gaze estimation from low-resolution videos. At the core of our approach is a neural network that classifies gaze directions in real time. We compared our method, called iCatcher, to manually annotated videos from a prior study in which infants looked at one of two pictures on a screen. We demonstrated that the accuracy of iCatcher approximates that of human annotators and that it replicates the prior study's results. Our method is publicly available as an open-source repository at https://github.com/yoterel/iCatcher.

Video-based motion-resilient reconstruction of three-dimensional position for functional near-infrared spectroscopy and electroencephalography head mounted probes

Significance: We propose a video-based, motion-resilient, and fast method for estimating the position of optodes on the scalp. Aim: Measuring the exact placement of probes (e.g., electrodes and optodes) on a participant's head is a notoriously difficult step in acquiring neuroimaging data from methods that rely on scalp recordings (e.g., electroencephalography and functional near-infrared spectroscopy) and is particularly difficult for any clinical or developmental population. Existing methods of head measurements require the participant to remain still for a lengthy period of time, are laborious, and require extensive training. Therefore, a fast and motion-resilient method is required for estimating the scalp location of probes. Approach: We propose an innovative video-based method for estimating the probes' positions relative to the participant's head, which is fast, motion-resilient, and automatic. Our method builds on capitalizing the advantages and understanding the limitations of cutting-edge computer vision and machine learning tools. We validate our method on 10 adult subjects and provide proof of feasibility with infant subjects. Results: We show that our method is both reliable and valid compared to existing state-of-the-art methods by estimating probe positions in a single measurement and by tracking their translation and consistency across sessions. Finally, we show that our automatic method is able to estimate the position of probes on an infant head without lengthy offline procedures, a task that has been considered challenging until now. Conclusions: Our proposed method allows, for the first time, the use of automated spatial co-registration methods on developmental and clinical populations, where lengthy, motion-sensitive measurement methods routinely fail.

Lead concentrations and isotopic ratios in the sediments of the Sea of Galilee

The isotopic composition and concentrations of Pb in the sediments of the Sea of Galilee (Lake Kinneret) were measured. The studied sediments have been deposited in the lake since the early 1900s (ca. 1920), hence Pb data record the transition from a period when the lake vicinity was sparsely populated to the present (approximately 100,000 people living in the area around the lake). In general, there is either a constant or a relatively slow increase in Pb concentrations from 40 cm depth (3.5-4.4 microg/g; ca. 1920) to 17 +/- 2 cm below the sediment-water interface (3.7-7.2 microg/g;), which was deposited in the mid-1960s. From 17 +/- 2 cm below the surface, there is a much faster increase up to 7 +/- 2 cm below the surface (from 6.5 to 11.5 microg/g; 1982-1983), and from 7 +/- 2 cm there is a gradual decrease in Pb concentrations toward the sediment-water interface. At station G, near the outlet of the Jordan River, Pb concentrations drop between 29 and 25 cm below the surface, probably reflecting changes in the particulate load of the Jordan River due to the drying out of the Hula Swamp in the early 1950s. 206Pb/207Pb values in all the stations record most of the shifts displayed by Pb concentrations in the sediment. The estimated value of total Pb deposited annually in the lake sediment in the early 1990s is very close to the value obtained from measurements of Pb fluxes to the lake from eolian and fluvial sources. On the basis of the linear relationship between 206Pb/207Pb (or 208Pb/207Pb) and 1/[Pb], we argue that two end-members contribute most of the Pb to the lake sediments. Sources of Pb to the lake include (i) the weathering of basalt from the eastern Galilee and the Golan Heights contributing 2.6 +/- 0.5 microg/g Pb to the sediment and (ii) anthropogenic Pb that is affecting both surface and deep (from 30 to 40 cm) lake sediments. At station S, a third source, Pb released from soils developed on carbonates, should be considered as well.

Mechanisms and velocities of anthropogenic Pb migration in Mediterranean soils

The isotopic composition of Pb measured in soil samples was used to determine rates and mechanisms of anthropogenic Pb migration in the soil. Petrol-Pb found in soluble halogenated aerosols migrates into the soil and is retained in the soil by the stationary soil particles. Lead infiltration velocity is approximately 5 x 10(-1) cm/year, and its retardation factor is estimated to be on the order of 1 x 10(3). The infiltration of Pb into the soil is best described by the advection-dispersion equation under the assumption that the time scale of the longitudinal dispersion is much longer than the time scale of advection. Therefore, the contribution of dispersion to the solution of the advection-dispersion equation is negligible. As a result, the soil profile of petrol-Pb resembles the time-dependent input function of petrol-Pb. The estimated petrol-Pb penetration velocity and the isotopic composition profile of Pb in off-road soil are used for the computation of the fraction of anthropogenic Pb in this soil. It is calculated that the fraction of anthropogenic Pb in the acid-leached soil samples and in the soil residue of this soil profile drops from 60 and 22% near the surface to 6 and 0% at a depth of 33 cm, respectively. The downward migration velocity of Pb in soils of the studied area, which are typically 50 to 100 cm deep, implies a residence time of Pb in the soil of 100 to 200 years.