Assisted exploration of difficult transitions in humanoid motion tracking.

Authors

Chaojie Fu, Chengkai Su, Lei Jiang, Chengwei Zhuo, Kaixin Lan, Shaowen Cheng, Hongtao Wang

Published in

Frontiers in neurorobotics. Volume 20. Pages 1920711. Epub Sep 30, 2026.

Abstract

Humanoid motion tracking provides a scalable interface for transferring kinematic human motion to whole-body robot control, but highly dynamic and contact-rich references remain difficult because underactuation and discontinuous contacts can restrict the closed-loop state distributions explored during training. External wrenches can reduce dynamic mismatch during imitation, but retaining such assistance in the final controller can compromise physical realism. This paper proposes residual-wrench-assisted reinforcement learning (RWARL), a two-stage framework that uses base residual wrenches as a temporary optimization scaffold. RWARL first solves a relaxed tracking problem with residual-wrench assistance and then anneals the residual-wrench strength while adapting early-termination bounds and segment sampling to observed training difficulty. To keep the policy adaptable as the training distribution shifts from assisted to unassisted dynamics, RWARL incorporates plasticity-preserving mechanisms during relaxation. Experiments on highly dynamic and contact-rich references show that RWARL generally improves global tracking accuracy, with the largest gains near difficult transitions where standard training tends to produce conservative behavior. Compared with the baseline, the final unassisted RWARL policies reduce global body-position tracking error by 8.6%-37.8% on the highly agile and contact-rich motions examined in this work and achieve the largest successful-frame-rate gain of 41.8 percentage points on jump twist. A closed-loop predecessor-state diagnostic further links these gains to more favorable predecessor-state distributions around critical phases. We deploy the final unassisted controllers on Unitree G1 hardware, demonstrating that the learned policies remain executable after residual assistance is removed. Together, these findings suggest that RWARL uses residual wrenches to reshape learning around difficult, low-reachability transitions, allowing the final unassisted policy to inherit better tracking behavior in phases that standard training tends to underexplore.

PMID:
42876931
Bibliographic data and abstract were imported from PubMed on 15 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.