HuG — Learning Humanoid Whole-Body Affordance-Aware Grasping with Active Perception

HuG: Learning Humanoid Whole-Body
Affordance-Aware Grasping
with Active Perception

Xuanbin Peng^*§, Lai Wei^*, Yuze Zhai^*, Qingyuan Yu^*, Keyi Wang, Jianglong Ye^†, Xiaolong Wang^† ^*Equal contribution. ^§Project lead. ^†Equal advising. University of California, San Diego CoRL 2026Conference on Robot Learning

pdf arxiv code live demo

TLDR: One policy lets a humanoid grasp anywhere from a deep squat to a full reach, with zero demonstrations. It is trained only on kinodynamically feasible grasps: 73% of kinematically valid candidates fail under the real whole-body controller and are discarded. 87.5% real-world success, zero-shot on unseen objects and scenes.

Interactive Demo

Every grasp runs live in your browser. Nothing is pre-rendered.

Try It Yourself ↗

Technical Highlights

Whole-Body–Hand Grasp Co-optimization

Tabletop grasp synthesis poses a hand on a fixed body. We solve hand and body configuration together, so the policy learns whole-body coordination across the full workspace.

Rendered sweep. Training randomizes torso height continuously, so no shelf level is ever out of distribution.

Kinodynamic-Aware Affordance Learning at Scale

On a humanoid, reachable ≠ executable. Affordances are learned only from synthetic grasps that execute under the deployed controller, so they are kinodynamically feasible by construction.

Filtering at scale. 73% of kinematically valid grasps are not dynamically feasible; the 600k that survive are the training set.

Seventeen renders of the humanoid grasping from a table at 0.10 m, in a deep squat, up to 0.90 m at full extension

One policy across table heights 0.10–0.90 m, deep squat to full extension; training covered only 0.40–0.90 m.

Full-Pipeline Shelf Picking

Perceive → Align → Grasp on a multi-level shelf: find the target from a language instruction, align the whole body, grasp.

One continuous run across shelves and objects, including objects unseen in training. Insets show the robot's head camera; use the arrows to switch runs.

Continuous Whole-Body Grasping

Language-Guided Grasping

The same table seen from a second camera, with the five
queried objects boxed in the order they are grasped.

A VLM orders the objects into a grasp plan (left); the robot executes it one language query at a time (right). Highlighted box: current target.

One Policy Across Diverse Real-World Scenes

Hallway

Kitchen Table

Office Lounge

Window Bench

Lab Table (black)

Kitchen Counter

Dining Room

Lab Table (yellow)

Meeting Room

Lounge Bench

Sink Counter

Round Table

Method

Top: deployment. Bottom: training, from synthetic scenes through kinodynamic filtering to the point-cloud policy.

BibTeX

@inproceedings{peng2026hug,
  title     = {Learning Humanoid Whole-Body Affordance-Aware Grasping with Active Perception},
  author    = {Peng, Xuanbin and Wei, Lai and Zhai, Yuze and Yu, Qingyuan and Wang, Keyi and Ye, Jianglong and Wang, Xiaolong},
  booktitle = {Conference on Robot Learning (CoRL)},
  year      = {2026}
}