RLDX-1.1-FT-ROBOCASA

RLDX-1.1 overview

RLDX-1.1 is a Vision-Language-Action model (VLA) for human-like dexterous manipulation. Beyond the versatile intelligence inherited from pre-trained VLM backbones, RLDX-1.1 adds three functional capabilities — motion awareness, long-term memory, and physical sensing — through a unified Multi-Stream Action Transformer (MSAT) architecture, a synthetic-augmented training pipeline, and a real-time inference stack.

This repository hosts RLDX-1.1-FT-ROBOCASA, RLWRLD/RLDX-1.1-PT fine-tuned on RoboCasa Kitchen (24 tasks) with 4-frame video input.

What's New in RLDX-1.1

Better performance. A refined RLDX-1 corpus plus a broad collection of open-source robot datasets gives a stronger pre-training foundation. Video-input fine-tunes enable the motion module, which RLDX-1 video checkpoints did not use.
Smoother actions in low DoF. Padded action dimensions are masked from pre-training on, so flow-matching noise no longer reaches unused joints. Keep mask_padded_action_dims=True when fine-tuning from this checkpoint.
Faster training and inference. Training is 2.27× faster than RLDX-1. Inference is up to 9.4× faster on H100 and 3.7× on RTX 5090, with fp8 on every supported GPU and nvfp4 on Jetson Thor, GB10 and RTX 5090. See Optimization.

Details are in the README.

RLDX-1.1 architecture

Model Family

Video input

Checkpoint	Type	Description
`RLDX-1.1-PT`	Base	Pre-trained
`RLDX-1.1-FT-ROBOCASA` (this repo)	Fine-tuned	RoboCasa Kitchen
`RLDX-1.1-FT-GR1`	Fine-tuned	GR-1 Tabletop
`RLDX-1.1-FT-LIBERO`	Fine-tuned	LIBERO
`RLDX-1.1-FT-RC365`	Fine-tuned	RoboCasa365

Image input

Checkpoint	Type	Description
`RLDX-1.1-PT-IMG`	Base	Pre-trained
`RLDX-1.1-FT-ROBOCASA-IMG`	Fine-tuned	RoboCasa Kitchen
`RLDX-1.1-FT-GR1-IMG`	Fine-tuned	GR-1 Tabletop
`RLDX-1.1-FT-LIBERO-IMG`	Fine-tuned	LIBERO
`RLDX-1.1-FT-RC365-IMG`	Fine-tuned	RoboCasa365

RLDX-1 checkpoints also load with this codebase; see Model Checkpoints.

Performance

Benchmark	Success Rate (%)
RoboCasa Kitchen	72.3

Evaluation guide: RoboCasa Kitchen. Training recipe: finetune_robocasa.sh with VARIANT=video.

The comparison with other VLAs is in the README.

Quick Start

git clone --branch v1.1.0 https://github.com/RLWRLD/RLDX-1.git
cd RLDX-1
uv sync --python 3.10

Load a checkpoint and predict an action chunk in-process:

from rldx.serving.policy import load_policy
from rldx.data.embodiment_tags import EmbodimentTag

policy = load_policy(
    model_path="RLWRLD/RLDX-1.1-FT-ROBOCASA",
    embodiment_tag=EmbodimentTag.GENERAL_EMBODIMENT,
    device="cuda:0",
)

# observation: {"video": {...}, "state": {...}, "language": {...}}
actions, info = policy.get_action(observation)

Serve the same policy over ZeroMQ for simulator rollouts or a real robot:

uv run python rldx/eval/run_rldx_server.py \
    --model-path RLWRLD/RLDX-1.1-FT-ROBOCASA \
    --embodiment-tag GENERAL_EMBODIMENT \
    --host 0.0.0.0 --port 20000

Runtime profiles (fp8 / nvfp4, CUDA graphs) and Real-Time Chunking are described in docs/inference_server.md.

Model Details

Architecture: Multi-Stream Action Transformer (MSAT) with a Qwen3-VL vision-language backbone and cognition tokens, trained with flow matching.
Inputs: RGB video, 4 frames per camera (stride 2), proprioceptive state, and a language instruction.
Outputs: action chunks of 16 steps.
Modules: motion module on (inserted at VLM layer 9); memory and physics off. Padded action dimensions are masked (mask_padded_action_dims=True).
Embodiment tag: GENERAL_EMBODIMENT.
Base model: RLWRLD/RLDX-1.1-PT.
Backbone: initialized from Qwen/Qwen3-VL-8B-Instruct and modified through further training (see Third-Party Notice).
Parameters: 6.6B.

A walkthrough of the architecture is in docs/architecture.md.

Intended Use and Limitations

Intended use. Research on robotic manipulation, fine-tuning on custom embodiments, simulation benchmarking, and non-commercial real-robot deployment under the RLWRLD Model License v2.0. Commercial use requires prior written permission from RLWRLD (§3.1).

Out of scope. Surveillance or monitoring of individuals without their consent, and any use that violates applicable laws or regulations (§3.5 of LICENSE.md).

Limitations. Results are reported on the RoboCasa Kitchen benchmark. Other scenes, objects, or embodiments are not guaranteed; for a different robot or dataset, fine-tune from RLWRLD/RLDX-1.1-PT instead.

Third-Party Notice

The vision-language backbone of this checkpoint was initialized from Qwen3-VL-8B-Instruct by the Qwen Team, Alibaba Cloud, licensed under the Apache License 2.0, and was modified by RLWRLD through further training. See NOTICE.

Citation

@article{rldx2026,
  title={RLDX-1 Technical Report},
  author={Dongyoung Kim and Huiwon Jang and Myungkyu Koo and Suhyeok Jang and Taeyoung Kim and others},
  year={2026},
  journal={arXiv preprint arXiv:2605.03269},
  eprint={2605.03269},
  archivePrefix={arXiv}
}

Please also cite the Qwen3-VL backbone and RoboCOIN, whose data was used in pre-training.

@misc{qwen3technicalreport,
      title={Qwen3 Technical Report},
      author={Qwen Team},
      year={2025},
      eprint={2505.09388},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.09388},
}

@article{robocoin,
    title={RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation},
    author={Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, Zhaoye Long, Yue Wang, Chong Liu, Dihan Wang, Ziqiang Ni, Xiang Yang, You Liu, Ruoxuan Feng, Runtian Xu, Lei Zhang, Denghang Huang, Chenghao Jin, Anlan Yin, Xinlong Wang, Zhenguo Sun, Junkai Zhao, Mengfei Du, Mingyu Cao, Xiansheng Chen, Hongyang Cheng, Xiaojie Zhang, Yankai Fu, Ning Chen, Cheng Chi, Sixiang Chen, Huaihai Lyu, Xiaoshuai Hao, Yequan Wang, Bo Lei, Dong Liu, Xi Yang, Yance Jiao, Tengfei Pan, Yunyan Zhang, Songjing Wang, Ziqian Zhang, Xu Liu, Ji Zhang, Caowei Meng, Zhizheng Zhang, Jiyang Gao, Song Wang, Xiaokun Leng, Zhiqiang Xie, Zhenzhen Zhou, Peng Huang, Wu Yang, Yandong Guo, Yichao Zhu, Suibing Zheng, Hao Cheng, Xinmin Ding, Yang Yue, Huanqian Wang, Chi Chen, Jingrui Pang, YuXi Qian, Haoran Geng, Lianli Gao, Haiyuan Li, Bin Fang, Gao Huang, Yaodong Yang, Hao Dong, He Wang, Hang Zhao, Yadong Mu, Di Hu, Hao Zhao, Tiejun Huang, Shanghang Zhang, Yonghua Lin, Zhongyuan Wang and Guocai Yao},
    journal={arXiv preprint arXiv:2511.17441},
    url = {https://arxiv.org/abs/2511.17441},
    year={2025}
}

License

Released under the RLWRLD Model License v2.0, a non-commercial license with attribution and share-alike terms. See LICENSE.md for the full text. By using this model you agree to those terms, including the use restrictions in §3.5. The Qwen3-VL component remains subject to the Apache License 2.0 as described in NOTICE.