srlwam — Cosmos Edge policies

Cosmos Edge policies for Franka Panda two-cube stacking (stack_v3) and PushT (pusht_v3), trained in simulation and fine-tuned on real teleoperated recordings, packaged for deployment on the real robot. Training data: dataset repo aabyaneh/srlwam. The earlier stack_v1/v2 checkpoints were removed (2026-10-21); they remain in this repo's git history.

Deploying? Read AGENTS.md first, then follow deploy/DEPLOYMENT.md gate by gate. The checkpoints do not all expect the same robot state; the deploy code handles it from each checkpoint's deploy.json, and the sanity checks verify it before the arm moves.

Cosmos Edge PushT v3

Cosmos3 Edge checkpoints for PushT v3 (push the T block to the target outline with a clamped pen): sim only from DROID, then real supervised and real video-only fine-tunes on 16 real teleoperation episodes (the stack_v3 count; 10 held out). Same recipe as stack_v3 (stride 1, 10 Hz) but with proprioception: flange pose + fingers in the robot base frame, which the PushT v3 sim shares with the real robot (start poses agree to 1.5 mm / 0.4 deg).

Conventions: gripper +1 = CLOSE (constant; the pen is held). One action unit = 0.392 m of realised flange motion per step: the PushT v3 sim controller realises ~39% of each command, so real recordings were scaled x2.40 and real commands = action / 2.40 (command.real_controller_delta). PushT v3 sim demos move ~5x slower per step than the real teleop. The stack fixture self-test (check_cosmos.py) does not apply to these bundles.

Held-out validation logged during training (sim run: held-out sim; real runs: 10 held-out real episodes); normalized action MAE in PushT v3's own statistics. Gripper accuracy is omitted: the gripper is constant.

run	checkpoint	step	role	action MAE	position MAE (mm, sim units)	flow loss
sim only (PushT v3 sim from Cosmos3-Edge-Policy-DROID)	sim_pusht_v3_droid	1750	best (final 2000: 0.0457)	0.0424	0.67	0.4907
real supervised from sim (16 real episodes)	real_supervised_from_sim_pusht_v3	4000	best = final	0.2546	3.78	1.7152
real video-only from sim (16 real episodes, no real action labels)	real_video_from_sim_pusht_v3	4000	final (flow-loss minimum 3000)	0.3276	4.41	0.4525

The video-only run's action score is a diagnostic (no real labels) and stays flat over training. Recorded-action agreement scores, not closed-loop robot success rates.

Cosmos Edge stack_v3

Cosmos3 Edge checkpoints trained on the stack_v3 simulation data (stride 1, 10 Hz, sim actions with exactly zero rotation) with no proprioception: the robot state is never fed. Three runs, one bundle each: sim only, real supervised from sim, and real video-only from sim (real camera video, no real action labels). Real data is the same 16 train / 4 held-out recordings, rescaled to stack_v3 units. The real-only baseline was not retrained: the stack_v2 real-only bundles it pointed to were removed (2026-10-21).

stack_v3 conventions differ from v1/v2: gripper +1 = CLOSE, action gain 0.9877 m/unit, real controller units = action / 0.9545, rotation identically 0. The shared adapter reads all of these from deploy.json. It now also honours action.real_controller_scale, which makes command.real_controller_delta correct for the sim_v2-unit checkpoints too.

Dense open-loop evaluation on 906 held-out real windows (episodes 0, 1, 15, 17), normalized MAE (lower is better; the error scale differs from stack_v2, so compare within this table). "Rotation set to 0" clamps predicted rotation, as the adapter does on the robot.

run	checkpoint	role	overall	rotation set to 0	position	gripper	gripper acc.	dir. cosine (step 0)
sim only (stack_v3 sim from Cosmos3-Edge-Policy-DROID)	sim_v3_droid	best = final (sim MAE 0.0326)	0.118	0.114	0.182	0.248	88.6%	+0.297
real supervised from sim	real_supervised_from_sim_v3	best = final (MAE 0.0704)	0.086	0.085	0.144	0.163	91.9%	+0.362
real video-only from sim (no real action labels)	real_video_from_sim_v3	final	0.106	0.084	0.125	0.213	91.2%	+0.367

With rotation clamped, video-only matches supervised overall and is better on position; the remaining gap is the gripper. real_video_from_sim_v3 emits a constant ~+0.025 rad/step rotation that must be dropped (the adapter does). Recorded-action agreement scores, not closed-loop robot success rates.

Cosmos Edge stack_v3 with proprioception

The stack_v3 runs retrained with proprioception (9-D flange pose + fingers, raw_robot frame; quaternion span floored at 0.05). Everything else (data, splits, units, gripper +1 = CLOSE, real commands = action / 0.9545) is as in the no-proprio stack_v3 bundles above, so the two sets compare directly. Also: supervised fine-tunes on exactly 1 and 2, 5 and 10 real episodes. Every run also has <bundle>_iter<step> checkpoints (every 500 steps) for selection. No fixture references or dense evaluation yet; scores are training-time held-out validation (sim run: held-out sim; real runs: episodes 0, 1, 15, 17).

run	checkpoint	step	role	action MAE	position MAE (mm)	flow loss
sim only (stack_v3 sim, with proprio)	sim_v3_proprio_droid	2000	best = final	0.0337	0.44	0.5915
real supervised, 16 real episodes	real_supervised_from_sim_v3_proprio	2000	best	0.0801	1.27	0.8093
real video-only (no real labels)	real_video_from_sim_v3_proprio	4000	final	0.1010	1.07	0.7530
real supervised, exactly 1 real episode	real_supervised_n1_from_sim_v3_proprio	4000	best = final	0.1704	1.31	2.9113
real supervised, exactly 2 real episodes	real_supervised_n2_from_sim_v3_proprio	500	best	0.1313	1.31	1.1184
real supervised, exactly 5 real episodes	real_supervised_n5_from_sim_v3_proprio	3000	best	0.0704	1.16	1.2636
real supervised, exactly 10 real episodes	real_supervised_n10_from_sim_v3_proprio	3000	best	0.0698	1.12	1.0533

The video-only run's action score is a diagnostic (no real labels). Recorded-action agreement scores, not closed-loop robot success rates.

Intermediate checkpoints for selection (stack_v3)

Every 500 steps of the six runs above (sim 500-2000, real 500-4000), plus the best-validation step where it falls between (v2 no-proprio supervised 3750, v3 video-only 1250). Bundles are <release bundle>_iter<step> beside the release bundle; the release bundles themselves are the final steps. Scores are the held-out validation logged during training (sim runs: held-out sim; real runs: held-out real episodes 0/1/15/17), normalized action MAE in each run's own normalization, so compare steps within a run; flow loss is the held-out training-objective loss. Video-only runs never used real labels: select them on flow loss or on your own evaluation; their action scores are diagnostic.

run	step	bundle	action MAE	gripper acc.	flow loss
stack_v3 sim	500	sim_v3_droid_iter500	0.0601	0.961	0.7767
stack_v3 sim	1000	sim_v3_droid_iter1000	0.0405	0.974	0.6276
stack_v3 sim	1500	sim_v3_droid_iter1500	0.0377	0.975	0.5912
stack_v3 sim	2000 (release)	sim_v3_droid	0.0326	0.977	0.5872
stack_v3 real supervised	500	real_supervised_from_sim_v3_iter500	0.0960	0.917	0.8330
stack_v3 real supervised	1000	real_supervised_from_sim_v3_iter1000	0.1056	0.943	0.7462
stack_v3 real supervised	1500	real_supervised_from_sim_v3_iter1500	0.0764	0.958	0.6446
stack_v3 real supervised	2000	real_supervised_from_sim_v3_iter2000	0.0827	0.948	0.6952
stack_v3 real supervised	2500	real_supervised_from_sim_v3_iter2500	0.0824	0.964	0.7451
stack_v3 real supervised	3000	real_supervised_from_sim_v3_iter3000	0.0789	0.958	0.9187
stack_v3 real supervised	3500	real_supervised_from_sim_v3_iter3500	0.0759	0.964	0.8860
stack_v3 real supervised	4000 (release)	real_supervised_from_sim_v3	0.0704	0.979	0.9209
stack_v3 real video-only	500	real_video_from_sim_v3_iter500	0.0892	0.958	0.5597
stack_v3 real video-only	1000	real_video_from_sim_v3_iter1000	0.0970	0.932	0.5428
stack_v3 real video-only	1250	real_video_from_sim_v3_iter1250	0.0996	0.917	0.5246
stack_v3 real video-only	1500	real_video_from_sim_v3_iter1500	0.0874	0.958	0.5367
stack_v3 real video-only	2000	real_video_from_sim_v3_iter2000	0.0973	0.948	0.5848
stack_v3 real video-only	2500	real_video_from_sim_v3_iter2500	0.0917	0.958	0.6563
stack_v3 real video-only	3000	real_video_from_sim_v3_iter3000	0.0933	0.964	0.6972
stack_v3 real video-only	3500	real_video_from_sim_v3_iter3500	0.0962	0.958	0.7366
stack_v3 real video-only	4000 (release)	real_video_from_sim_v3	0.0956	0.958	0.7536