Early Attraction in Temporally Controlled Sight Reading of Music

The influence of executive functions on eye-hand span and piano performance during sight-reading

The ability to perform an unrehearsed piece of music, or sight-read, is a skill required by music performers. In sight-reading, the performer reads and plays the music simultaneously, which requires the coordination of visual, auditory, and motor processing. While performing, they display a characteristic called eye-hand span, in which the part of the score being looked at precedes the part being played. They must recognize, decipher, and process the score in the time between reading a note and playing it. An individual's executive function (EF) that control their cognition, emotions, and behavior may be involved in overseeing these individual movements. However, no study has investigated how EF is related to the eye-hand span and performance during sight-reading. Therefore, the purpose of this study is to clarify the relationships among EF, eye-hand span, and piano performance. Thirty-nine Japanese pianists and college students aspiring to be pianists with an average of 33.3 years of experience participated in this study. They performed sight-reading of two types of music scores with different difficulty levels while their eye movements were measured using an eye tracker to obtain their eye-hand span. The core EFs of inhibition, working memory, and shifting, were measured directly for each participant. Piano performance was evaluated by two pianists who did not participate in the study. Structural equation modeling was used to analyze the results. The results showed that auditory working memory predicted eye-hand span (β = .73, p < .001 in easy score; β = .65, p < .001 in difficult score), and eye-hand span predicted performance (β = .57, p < .001 in easy score; β = .56, p < .001 in difficult score). Auditory working memory did not directly affect performance, but through eye-hand span. The eye-hand span for easy scores was significantly greater than that for difficult scores. Furthermore, in a difficult music score, the shifting ability predicted higher piano performance. These suggest that the input of notes from the eyes becomes sound in the brain and activates the auditory working memory, which is then transmitted to finger movement, resulting in piano performance. In addition, it was suggested that shifting ability is also needed to perform difficult scores.

Review on Eye-Hand Span in Sight-Reading of Music

In a sight-reading task, the position of the eyes on the score is generally further ahead than the note being produced by the instrument. This anticipation allows musicians to identify the upcoming notes and possible difficulties and to plan their gestures accordingly. The eye-hand span (EHS) corresponds to this offset between the eye and the hand and measures the distance or latency between an eye fixation on the score and the production of the note on the instrument. While EHS is mostly quite short, the variation in its size can depend on multiple factors. EHS increases in line with the musician's expertise level, diminishes as a function of the complexity of the score and can vary depending on the context in which it is played. By summarizing the main factors that affect EHS and the methodologies used in this field of study, the present review of the literature highlights the fact that a) to ensure effective sight reading, the EHS must be adaptable and optimized in size (neither too long not too short), with the best sight readers exhibiting a high level of perceptual flexibility in adapting their span to the complexity of the score; b) it is important to interpret EHS in the light of the specificities of the score, given that it varies so much both within and between scores; and c) the flexibility of EHS can be a good indicator of the perceptual and cognitive capacities of musicians, showing that a musician's gaze can be attracted early by a complexity in a still distant part of the score. These various points are discussed in the light of the literature on music-reading expertise. Promising avenues of research using the eye tracking method are proposed in order to further our knowledge of the construction of an expertise that requires multisensory integration.

The association of eye movements and performance accuracy in a novel sight-reading task

The present study investigated how eye movements were associated with performance accuracy during sight-reading. Participants performed a complex span task in which sequences of single quarter note symbols that either enabled chunking or did not enable chunking were presented for subsequent serial recall. In between the presentation of each note, participants sight-read a notated melody on an electric piano in the tempo of 70 bpm. All melodies were unique but contained four types of note pairs: eighth-eighth, eighthquarter, quarter-eighth, quarter-quarter. Analyses revealed that reading with fewer fixations was associated with a more accurate note onset. Fewer fixations might be advantageous for sight-reading as fewer saccades have to be planned and less information has to be integrated. Moreover, the quarter-quarter note pair was read with a larger number of fixations and the eighth-quarter note pair was read with a longer gaze duration. This suggests that when rhythm is processed, additional beats might trigger re-fixations and unconventional rhythmical patterns might trigger longer gazes. Neither recall accuracy nor chunking processes were found to explain additional variance in the eye movement data.

Automated Page Turner for Musicians

An increasing number of musicians are opting to use tablet devices instead of traditional print media for their music sheets since the digital medium offers the benefit of storing a lot of music in a compact space. The limited screen size of the tablet devices makes the music difficult to read and musicians often opt to display part of the music page at a time. With fewer music lines on display, the musician will then have to resort to scrolling through the music to read the entire score. This scrolling is annoying since the musicians will need to remove their hands from the instrument to interact with the tablet, causing a break in the music if this is not done quickly enough, or if the tablet is not sufficiently responsive. In this paper, we describe an alternative page turning system which automates the page turning event of the musician. By actively monitoring the musician's on-screen point of regard, the system retains the musician in the loop and thus, the page turns are attuned to the musician's position on the score. By analysing the way the musician's gaze changes between attention to the score and the instrument as well as the way musicians fixate on different parts of the score, we note that musicians often look away from the score and toward their hands, or elsewhere, when playing the instrument. As a result, the eye regions fall outside the field-of-view of the eye-gaze tracker, giving rise to erratic page-turns. To counteract this problem, we create a gaze prediction model that uses Kalman filtering to predict where the musician would be looking on the score. We evaluate our hands-free page turning system using 15 different piano songs containing different levels of difficulty, various repeats, and which also required playing in different registers on the piano, thus, evaluating the applicability of the page-turner under different conditions. Performance of the page-turner was quantified through the number of correct page turns, the number of delayed page turns, and the number of mistaken page turns. Of the 289 page turns involved in the experiment, 98.3% were successfully executed, 1.7% were delayed, while no mistaken page turns were observed.

The application of eye-tracking in music research

Though eye-tracking is typically a methodology applied in the visual research domain, recent studies suggest its relevance in the context of music research. There exists a communityof researchers interested in this kind of research from varied disciplinary backgrounds scattered across the globe. Therefore, in August 2017, an international conference was held at the Max Planck Institute for Empirical Aesthetics in Frankfurt, Germany,to bring this research community together. The conference was dedicated to the topic of music and eye-tracking, asking the question: what do eye movements, pupil dilation, and blinking activity tell us about musical processing? This special issue is constituted of top-scoring research from the conference and spans a range of music-related topics. From tracking the gaze of performers in musical trios to basic research on how eye movements are affected by background music, the contents of this special issue highlight a variety of experimental approaches and possible applications of eye-tracking in music research.