Instructions to use AIGeeksGroup/WAM-Policy with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use AIGeeksGroup/WAM-Policy with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("robbyant/lingbot-va-base", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("AIGeeksGroup/WAM-Policy") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
WAM-Policy: World Action Models with On-Policy Distillation for Quadrupedal Loco-Manipulation
Weight repository for WAM-Policy checkpoints and prompt embeddings.
WAM-Policy: World Action Models with On-Policy Distillation for Quadrupedal Loco-Manipulation
Junlin Long*, Xu Deng*, Yiran Wang, Yue Yang†, Yang Zhao†, and Zeyu Zhang‡
*Equal contribution. †Corresponding authors. ‡Project lead.
Website | WAM-Policy code | QLM-Bench code | QLM-Bench dataset
✏️ Citation
Citation will be added after publication.
🏃 Introduction
Arm-free quadrupeds can perform loco-manipulation, walking on three legs while manipulating with a front leg, yet no existing policy for it follows language instructions. World action models (WAMs) have emerged as effective language-conditioned manipulation policies, but are pretrained for robot arms and sample slowly. WAM-Policy turns one WAM into a quadrupedal loco-manipulation policy via two stages. The first, Cross-Morphology Policy Adaptation and Acceleration, adapts the arm-centric WAM to quadrupeds and distills the transferred policy into a student with one video and two action steps by on-policy self-distillation (OPSD). This student, however, is bounded by its teacher, which only imitates demonstrations. The second stage, Task-Specific Improvement and Merging, therefore tunes the teacher by reinforcement learning (RL) and merges the slow specialists into one few-step student by multi-teacher on-policy distillation (M-OPD). On QLM-Bench, ten simulated tasks with 1,000 demonstrations synthesized without teleoperation, the transferred policy reaches 77.0%, and the OPSD student keeps 76.7% while sampling 14 times faster. Task-specific RL and M-OPD lift it to 85.0%. We also deploy the OPSD student on a real Unitree Go2 Air. WAM-Policy thus achieves a stronger quadrupedal policy at a fraction of the inference cost.
- Cross-morphology adaptation and acceleration. Stage 1 replaces only the visual and action interfaces of a WAM, fine-tunes it on quadruped demonstrations, and accelerates it by a few-step form of on-policy self-distillation (OPSD), in which the same network with more sampling steps labels the student's own samples. The transferred policy reaches 77.0%, and the OPSD student keeps 76.7% (69.3% without distillation) while sampling 14 times faster.
- Task-specific improvement and merging. Stage 2 tunes specialist teachers by per-task reinforcement learning and merges them by multi-teacher on-policy distillation (M-OPD), which routes each context to the teacher of its task. On pushing a box and opening a door, the specialists succeed in every episode, against 47% and 53% for the OPSD teacher, and the M-OPD student reaches 90% and 87%, raising the ten-task success from 76.7% to 85.0%, above the OPSD teacher (77.0%).
- QLM-Bench, ten simulated loco-manipulation tasks with 1,000 language-annotated demonstrations synthesized without teleoperation, and evaluation in simulation and on a real Unitree Go2 Air, where the OPSD student runs without real-robot data.
Method
QLM-Bench
See the QLM-Bench code README for data preparation and evaluation setup, and the QLM-Bench dataset card for downloads, formats and splits.
Checkpoints
| Checkpoint | Folder | Weights | NFE (video / action) |
|---|---|---|---|
| Stage 1.1 teacher | stage1_1_teacher/ |
Full transformer in transformer/ (~18.9 GiB) |
25 / 50 |
| Stage 1.2 OPSD student (S300 EMA) | stage1_2_opsd_student/ |
student_adapter.pt |
1 / 2 |
| Stage 2.1 specialists | stage2_1_specialists/{push_box,open_room_door}/ |
One adapter.pt per task |
25 / 50 |
| Stage 2.2 M-OPD student (S300 EMA) | stage2_2_mopd_student/ |
student_adapter.pt |
1 / 2 |
| Arm OPSD student (S300 EMA) | supplementary/arm_libero_opsd_student/ |
student_adapter.pt |
1 / 2 |
Quadruped adapters load unmerged on the Stage 1.1 teacher. The 1/2-NFE baseline without distillation uses the same teacher weights. Arm adapters use the LingBot-VA LIBERO-Long base.
The checkpoint manifest lists file paths, sizes and SHA-256 digests. Each checkpoint
directory includes loader metadata (wam_policy.json for quadruped and arm_libero.json for LIBERO).
Keep prompt_embeddings/<task>/embedding.pt with its reference.json for each quadruped task.
See the QLM-Bench dataset card for the task instructions.
Ablation checkpoints
| Embodiment | Variant | Updates | Folder |
|---|---|---|---|
| Quadruped | Teacher cache only | 300, 2,000 | ablations/quadruped/teacher_cache_only/{s300_ema,s2000_ema}/ |
| Quadruped | Streaming only | 300 | ablations/quadruped/streaming_only/s300_ema/ |
| Quadruped | OPSD | 900 | ablations/quadruped/opsd/s900_ema/ |
| Quadruped | RGB + proprioception | 5,000 | ablations/quadruped/rgb_proprioception/s5000/ |
| Arm (LIBERO) | Teacher cache only | 300, 2,000 | ablations/arm_libero/teacher_cache_only/{s300_ema,s2000_ema}/ |
| Arm (LIBERO) | Streaming only | 300 | ablations/arm_libero/streaming_only/s300_ema/ |
| Arm (LIBERO) | Collected once | 300 | ablations/arm_libero/collected_once/s300_ema/ |
| Arm (LIBERO) | SFT only | 692 | ablations/arm_libero/sft_only/s692_ema/ |
| Arm (LIBERO) | OPSD | 900 | ablations/arm_libero/opsd/s900_ema/ |
The ablation adapters use EMA weights. The RGB + proprioception checkpoint is a full transformer evaluated at 25/50 NFE. See Tab. 6 in the code README's reproduction table for the evaluation entry point.
To download one ablation, select its folder along with any base model and prompts it requires.
huggingface-cli download AIGeeksGroup/WAM-Policy --revision main \
--include "ablations/quadruped/teacher_cache_only/s300_ema/*"
Upstream models
| Needed for | Repository | Revision |
|---|---|---|
| VAE and text encoder config for quadruped inference | robbyant/lingbot-va-base | 68b7bc1b35da6ddc67ea94c4ceb58d768fbb3f9c |
| Base model of the arm student | robbyant/lingbot-va-posttrain-libero-long | 0e89d1e753019988aba484e8da2dc0810e264d9f |
Usage
Download only what you need, pinned to this branch, and the LingBot-VA base (VAE and text-encoder config):
pip install -U "huggingface_hub[cli]"
ROOT=$(huggingface-cli download AIGeeksGroup/WAM-Policy --revision main --quiet \
--include "stage1_1_teacher/*" "stage2_2_mopd_student/*" "prompt_embeddings/*")
LINGBOT_VA_BASE=$(huggingface-cli download robbyant/lingbot-va-base \
--revision 68b7bc1b35da6ddc67ea94c4ceb58d768fbb3f9c --quiet)
Serve a policy over qlm-live-v1 (two 640×360 RGB views + instruction → a chunk of 16 9-D commands). This needs the
WAM-Policy code and the qlm_bench.live package from QLM-Bench (pip install -e "<QLM-Bench>[live]", or put
<QLM-Bench>/src on PYTHONPATH):
python -m wam_policy.inference.serve --policy mopd_student \
--teacher "$ROOT/stage1_1_teacher" --adapter "$ROOT/stage2_2_mopd_student" \
--base-model "$LINGBOT_VA_BASE" --prompt-root "$ROOT/prompt_embeddings" --port 8862
--policy accepts teacher, specialist, opsd_student and mopd_student. The default sampler comes from
wam_policy.json. Use the separate LIBERO evaluation guide
for arm checkpoints.
Use the WAM-Policy experiment configs for training and the reproduction guide for the Standard100 evaluation settings.
License
The checkpoints in this repository are released under the Apache License 2.0. The Stage 1.1 teacher adapts the LingBot-VA base with a 9-D action head and fine-tuning on QLM-Bench demonstrations, and the OPSD student, specialist teachers, and M-OPD student are LoRA adapters on that teacher. The arm student is a LoRA adapter on the LIBERO-Long base, and the original weights are available from the repositories in Upstream models.
- Downloads last month
- -
Model tree for AIGeeksGroup/WAM-Policy
Base model
robbyant/lingbot-va-base


