arxiv:2610.29997

DONIC: Generalist Dexterous Bimanual Tracking Control from Human Demonstrations

Published on Oct 21

· Submitted by

Ziang Cao on Oct 22

Authors:

Mingze Sun ,

Ziang Cao ,

Abstract

DONIC is a generalist bimanual tracker trained on diverse human-motion data that uses reinforcement learning with recycled demonstrations and per-hand rewards to achieve high success on unseen objects without fine-tuning.

Generated by thinkingmachines/Inkling-Small

Despite recent progress in dexterous reference tracking, existing models focus primarily on single-hand manipulation and a limited range of interactions, leaving the bimanual dexterous behaviors common in real life unaddressed. The scarcity of contact-rich bimanual manipulation data and the increased complexity of coordinating contacts make developing a generalist bimanual tracker particularly challenging. Therefore, we present 1) DONIC, the first generalist bimanual tracker jointly trained across operation categories and human-motion sources, together with 2) BiHOI, a multi-view dataset of 1,575 sequences spanning 229 objects and emphasizing sustained bimanual interaction, where reconstructed interactions are retargeted into robot references. DONIC combines proximal policy optimization with behavioral cloning in a feedback loop that recycles verified successful executions into training demonstrations. An object-centered per-hand reward maintains object-motion feedback while supervising each hand according to its reference contact role. Clip-Centered Advantages remove reference-dependent advantage offsets while preserving within-trajectory differences. Together, these designs enable one single policy to learn coordinated manipulation across heterogeneous references. Without finetuning, a single checkpoint achieves 86.79% success on unseen trajectories of training objects and 57.32% on held-out objects, surpassing the state-of-the-art bimanual tracking baselines by 77.36 and 45.73 percentage points.

View arXiv page View PDF Project page GitHub 20 Add to collection

Community

Caoza

Paper author Paper submitter about 14 hours ago

DONIC is a generalist dexterous bimanual tracking policy: one policy learns to track diverse human two-hand demonstrations with two robot hands, and generalizes to unseen trajectories and unseen objects without fine-tuning.

🔥 Highlights

🤲 Policy — One Policy, Two Hands. A single reference-conditioned policy controls two Wuji dexterous hands and tracks bimanual references across operation categories and human-motion sources.
📊 Data — BiHOI. 1,575 multi-view bimanual sequences over 229 objects and 14.77 hours of interaction, captured with a calibrated multi-view camera array and emphasizing sustained two-hand contact.
🧠 Learning — Signals Built for Two Hands. An object-centered per-hand reward, Clip-Centered Advantages, and a PPO + behavioral cloning loop that recycles replay-verified successful executions into training demonstrations.
🏆 Generalization — Zero Fine-Tuning. 86.79% success on unseen trajectories and 57.32% on unseen objects in simulation, 77.36 and 45.73 percentage points above the stronger of our bimanual adaptations of DexTrack and Dexplore.
📈 Scaling — More Human Data Helps. Adding BiHOI improves all seven metrics on both held-out splits, raising success to 88.68% and 59.76%.