Hiring in life sciences? Share your open positions with our professional community. Read more Close

Advertisement

HeteroGrad-DT: Gradient Flow Transformer for heterogeneous multi-task offline reinforcement learning.

Created on 12 Nov 2026

Authors

Ye Wang, Donghui Ma, Jiang Li, Lihong Guo

Published in

PloS one. Volume 21. Issue 11. Pages e0362298. Epub Nov 11, 2026.

Abstract

Offline reinforcement learning enables policy learning from static datasets without online exploration, making it particularly suitable for robotic control under safety and data constraints. Decision Transformer recasts offline reinforcement learning as conditional sequence modeling. Extending it to heterogeneous multi-task settings is challenging because task representations and dynamics differ, while joint training can suffer from destructive gradient interference. To address these challenges, we present HeteroGrad-DT, an integrated framework that employs task-specific tokenizers and decoupled prediction heads to retain task-native input and output dimensionality at the model interfaces while learning a shared sequence backbone. It further introduces a Gradient Flow Transformer that periodically coordinates cross-task updates using fixed-dimensional task statistics, compact summaries of shared-backbone gradients accumulated within the current synchronization interval, and an alignment-based coordination objective, rather than relying only on stepwise projection or static regularization. Experiments on the DeepMind Control Suite with up to 30 tasks show that HeteroGrad-DT improves average normalized return over representative multi-task methods. In the 30-task setting, it achieves a 32.3% relative improvement over standard training with a shared backbone and an 8.6% gain over a representative gradient-conflict mitigation baseline using the same task-aware interfaces. Ablation, mechanism, and sensitivity analyses further support complementary benefits from task-aware interfaces and gradient coordination in the evaluated settings.

PMID:
43013603
Bibliographic data and abstract were imported from PubMed on 12 Nov 2026.

Read full publication at:
Please sign in to see all details.

Advertisement

Stats

  • Community rating n/a 0 votes
  • Reviewers' rating n/a 0 votes
  • Your rating

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

  • Recommendations n/a n/a positive of 0 vote(s)
  • Views 6
  • Comments 0

Recommended by

  • No recommendations yet.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.

Advertisement