Language as a Universal Anchor for Cross-Domain Sketch-Based 3D Human Mesh Recovery.

Authors

Fei Wang, Zhineng Zhang, Junkun Jiang, Jingyun Liao, Hao Cai, Shanwen Guan, Zhifeng Hao

Published in

IEEE transactions on visualization and computer graphics. Volume PP. Oct 30, 2026. Epub Oct 30, 2026.

Abstract

When humans sketch the same pose, no two drawings are alike. Synthetic sketches exhibit algorithmic precision with clean edges and consistent strokes, while freehand sketches diverge wildly-each bearing the unique abstraction, style, and imperfections of its creator. This fundamental divergence has long challenged 3D human reconstruction systems, which struggle to bridge the chasm between these disparate visual domains. We present a paradigm shift: while sketches diverge, language converges. A pose described as "arms raised overhead" carries the same semantic meaning whether drawn by algorithm or artist. Building on this insight, we introduce a universal feature anchor-natural language-that remains constant across visual variations. Our framework leverages text descriptions to guide feature learning, creating cross-domain representations that transcend the synthetic-freehand divide. At the technical core lies our Text-based Body Pose Head (TBPH), featuring a novel gating mechanism where language-derived features dynamically reweight spatial regions of sketch features. This textguided attention enables the model to focus on semantically meaningful pose indicators while suppressing domain-specific noise and stylistic artifacts. By augmenting 26,000 sketch-pose pairs with rich textual descriptions, we enable cross-modal supervision that teaches our model to see past surface differences to underlying pose semantics. Extensive experiments demonstrate our method's superiority: with ideal descriptions, UniAnchor achieves 139.86 mm MPJPE on freehand sketches, a 4.5% improvement over TokenHMR, and further outperforms it by 11.08% in zero-shot generalization. With casual descriptions, our framework still retains a substantial portion of these gains, indicating that much of the language benefit is effectively internalized through multimodal training. More importantly, our model remains robust on highly abstract amateur sketches, demonstrating the practical value of multimodal supervision for cross-domain 3D human reconstruction.

PMID:
42961795
Bibliographic data and abstract were imported from PubMed on 31 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.