Emotion- and Performance-in-the-Loop: Multimodal Difficulty-Adaptive Exergame System-Controlled Between-Participants Study.

Authors

Liang Zhao, Dan Liu, Hanli Zhang, Guanci Yang, Donghua Zheng, Ling He

Published in

JMIR serious games. Volume 14. Pages e101438. Nov 05, 2026. Epub Nov 05, 2026.

Abstract

Existing exergames predominantly adjust difficulty based on explicit performance metrics, neglecting the player's implicit affective state, which often leads to early disengagement due to boredom or frustration.
To address this gap, we present a novel exergame system that implements joint emotion- and performance-driven dynamic difficulty adjustment (DDA), using a closed-loop controller that reasons over both motor performance and affective state.
The system integrates a computer-vision-based motion evaluation pipeline, a multimodal emotion recognition module that fuses facial action units (AUs), deep facial features, and motion-compensated heart rate variability (HRV) via an adaptive entropy-weighted decision-level fusion strategy, and a closed-loop difficulty controller. A controlled between-participants experiment (N=20) compared the adaptive system against a fixed-difficulty baseline during a 15-minute dance-exergame session.
Emotion recognition accuracy reached 81.07% (macro-F₁: 81.02%) for 7 emotional states. The adaptive group demonstrated significantly lower cognitive load (NASA-TLX [National Aeronautics and Space Administration Task Load Index], P<.05), higher positive affect (Positive and Negative Affect Schedule [PANAS], P<.05), greater system usability (System Usability Scale [SUS], P<.05), improved motion accuracy (82.5% vs 74.8%, P<.05), and a physiologically healthier training load indicated by higher HRV root mean square of successive differences (RMSSD; 34.6 vs 24.3 ms, P<.05).
The joint emotion-and-performance difficulty-adaptation mechanism is feasible and significantly improves user experience and training outcomes compared to a fixed-difficulty baseline.

PMID:
42987306
Bibliographic data and abstract were imported from PubMed on 06 Nov 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.