Pupil- and gaze dynamics track emotion content in naturalistic speech.

Authors

Sümeyye Şen-Alpay, Christian Keitel, Rosanne H Timmerman, Anne Keitel

Published in

Scientific reports. Jun 18, 2026. Epub Jun 18, 2026.

Abstract

Cortical tracking of speech features is a well-established marker of continuous speech processing, but far less is known about listeners' ocular responses to speech rhythm. Ocular responses are central to active sensing models in the auditory domain, where motor recruitment guides temporal speech prediction and attention allocation, potentially shaped by non-rhythmic cues, such as emotions. Here, we ask whether listeners' pupil response and eye movements track the acoustic speech signals and to what extent this tracking is modulated by emotion-related top-down factors, including subjective emotion ratings, mood, and trait empathy. In a validation study (N = 100), participants passively listened to two TED talks and intermittently rated speech segments on valence (negative-positive) and arousal (low-high). This suggested substantial variability in valence and arousal across speech segments in both talks. In the second study (N = 41), participants completed the same task while pupillometry and electrooculography (EOG) were recorded. Mutual information was used to quantify speech tracking in pupil dilation, along with horizontal and vertical eye movements. All ocular signals significantly tracked speech at low frequencies. High-arousal speech was associated with stronger pupil tracking but weaker vertical and horizontal EOG tracking. Negative speech valence was linked to stronger tracking in pupil and vertical eye-movement signals. Interactions between the speech-emotion dimensions, as well as their interactions with listeners' mood, further shaped these effects, giving rise to distinct patterns across ocular measures. Taken together, our findings provide evidence that ocular activity dynamically aligns to the temporal structure of natural speech and that this tracking is sensitive to both stimulus-driven and listener-dependent emotional factors.

PMID:
42315873
Bibliographic data and abstract were imported from PubMed on 19 Jun 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.