The Effect of Head Orientation on Perceived Gaze Direction: Revisiting Gibson and Pick (1963) and Cline (1967)

The integration of head and body cues during the perception of social interactions

Humans spend a large proportion of time participating in social interactions. The ability to accurately detect and respond to human interactions is vital for social functioning, from early childhood through to older adulthood. This detection ability arguably relies on integrating sensory information from the interactants. Within the visual modality, directional information from a person's eyes, head, and body are integrated to inform where another person is looking and who they are interacting with. To date, social cue integration research has focused largely on the perception of isolated individuals. Across two experiments, we investigated whether observers integrate body information with head information when determining whether two people are interacting, and manipulated frame of reference (one of the interactants facing observer vs. facing away from observer) and the eye-region visibility of the interactant. Results demonstrate that individuals integrate information from the body with head information when perceiving dyadic interactions, and that integration is influenced by the frame of reference and visibility of the eye-region. Interestingly, self-reported autistics traits were associated with a stronger influence of body information on interaction perception, but only when the eye-region was visible. This study investigated the recognition of dyadic interactions using whole-body stimuli while manipulating eye visibility and frame of reference, and provides crucial insights into social cue integration, as well as how autistic traits affect cue integration, during perception of social interactions.

The Time Course of Information Processing During Eye Direction Perception

Gaze directed at the observer (direct gaze) is a highly salient social signal. Despite the existence of a preferential orientation toward direct gaze, none of the studies carried out so far seem to have explicitly studied the time course of information processing during gaze direction judgment. In an eye direction judgment task, participants were presented with a sketch of a face. A temporal asynchrony was introduced between the presentation of the eyes and that of the rest of the face. Indeed, the face could be presented before the eyes, the eyes could be presented before the face, or the face and the eyes could be presented simultaneously. In a second time, the face direction was also manipulated. The results suggest that the time course of information processing during eye direction judgment follows a continuum that makes it possible to perceive the eyes first and then to use the facial context to judge the direction of gaze. Furthermore, the congruency between the direction of gaze and that of the face confirms this observation. Although these results are discussed in the light of existing theories about the mechanisms underlying gaze processing, our data provide new information suggesting that, despite their power to capture attention, the eyes probably have to stand out from a more general spatial configuration (i.e., the face) in order for their direction to be adequately processed.

Recognizing Students and Detecting Student Engagement with Real-Time Image Processing

With COVID-19, formal education was interrupted in all countries and the importance of distance learning has increased. It is possible to teach any lesson with various communication tools but it is difficult to know how far this lesson reaches to the students. In this study, it is aimed to monitor the students in a classroom or in front of the computer with a camera in real time, recognizing their faces, their head poses, and scoring their distraction to detect student engagement based on their head poses and Eye Aspect Ratios. Distraction was determined by associating the students’ attention with looking at the teacher or the camera in the right direction. The success of the face recognition and head pose estimation was tested by using the UPNA Head Pose Database and, as a result of the conducted tests, the most successful result in face recognition was obtained with the Local Binary Patterns method with a 98.95% recognition rate. In the classification of student engagement as Engaged and Not Engaged, support vector machine gave results with 72.4% accuracy. The developed system will be used to recognize and monitor students in the classroom or in front of the computer, and to determine the course flow autonomously.

The roles of gaze and head orientation in face categorization during rapid serial visual presentation

Little is known about how perceived gaze direction and head orientation may influence human categorization of visual stimuli as faces. To address this question, a sequence of unsegmented natural images, each containing a random face or a non-face object, was presented in rapid succession (stimulus duration: 91.7 ms per image) during which human observers were instructed to respond immediately to every face presentation. Faces differed in gaze and head orientation in 7 combinations - full-front views with perceived gaze (1) directed to the observer, (2) averted to the left, or (3) averted to the right, left ¾ side views with (4) direct gaze or (5) averted gaze, and right ¾ side views with (6) direct gaze or (7) averted gaze - were presented randomly throughout the sequence. We found highly accurate and rapid behavioural responses to all kinds of faces. Crucially, both perceived gaze direction and head orientation had comparable, non-interactive effects on response times, where direct gaze was responded faster than averted gaze by 48 ms and full-front view faster than ¾ side view also by 48 ms on average. Presentations of full-front faces with direct gaze led to an additive speed advantage of 96 ms to ¾ faces with averted gaze. The results reveal that the effects of perceived gaze direction and head orientation on the speed of face categorization probably depend on the degree of social relevance of the face to the viewer.

Task Dependent Effects of Head Orientation on Perceived Gaze Direction

The perception of gaze direction involves the integration of a number of sensory cues exterior to the eye-region. The orientation of the head is one such cue, which has an overall repulsive effect on the perceived direction of gaze. However, in a recent experiment, we found the measured effect of head orientation on perceived gaze direction differed within subjects, depending on whether a single- or two-interval task design was employed. This suggests a potential difference in the way the orientation of the head is integrated into the perception of gaze direction across tasks. Four experiments were conducted to investigate this difference. The first two experiments showed that the difference was not the result of some interaction between stimuli in the two-interval task, but rather, a difference between the types of judgment being made across tasks, where observers were making a directional (left/right) judgment in the single-interval task, and a non-directional (direct/indirect gaze) judgment in the two-interval task. A third experiment showed that this difference does not arise from observers utilizing a non-directional cue to direct gaze (the circularity of the pupil/iris) in making their non-directional judgments. The fourth experiment showed no substantial differences in the duration of evidence accumulation and processing between judgments, suggesting that observers are not integrating different sensory information across tasks. Together these experiments show that the sensory information from head orientation is flexibly weighted in the perception of gaze direction, and that the purpose of the observer, in sampling gaze information, can influence the consequent perception of gaze direction.

Context Modulates Congruency Effects in Selective Attention to Social Cues

Head and gaze directions are used during social interactions as essential cues to infer where someone attends. When head and gaze are oriented toward opposite directions, we need to extract socially meaningful information despite stimulus conflict. Recently, a cognitive and neural mechanism for filtering-out conflicting stimuli has been identified while performing non-social attention tasks. This mechanism is engaged proactively when conflict is anticipated in a high proportion of trials and reactively when conflict occurs infrequently. Here, we investigated whether a similar mechanism is at play for limiting distraction from conflicting social cues during gaze or head direction discrimination tasks in contexts with different probabilities of conflict. Results showed that, for the gaze direction task only (Experiment 1), inverse efficiency (IE) scores for distractor-absent trials (i.e., faces with averted gaze and centrally oriented head) were larger (indicating worse performance) when these trials were intermixed with congruent/incongruent distractor-present trials (i.e., faces with averted gaze and tilted head in the same/opposite direction) relative to when the same distractor-absent trials were shown in isolation. Moreover, on distractor-present trials, IE scores for congruent (vs. incongruent) head-gaze pairs in blocks with rare conflict were larger than in blocks with frequent conflict, suggesting that adaptation to conflict was more efficient than adaptation to infrequent events. However, when the task required discrimination of head orientation while ignoring gaze direction, performance was not impacted by both block-level and current trial congruency (Experiment 2), unless the cognitive load of the task was increased by adding a concurrent task (Experiment 3). Overall, our study demonstrates that during attention to social cues proactive cognitive control mechanisms are modulated by the expectation of conflicting stimulus information at both the block- and trial-sequence level, and by the type of task and cognitive load. This helps to clarify the inherent differences in the distracting potential of head and gaze cues during speeded social attention tasks.