ForceHarness: Turning Force into Action and Physics for Frozen Frontier Models
Abstract
Frontier multimodal models can now control robots zero-shot from images, yet many manipulation tasks depend on contact forces that images cannot show. Bringing force to an API-only frozen model raises two challenges. First, contact evolves much faster than an API model can decide, so the system must close the loop on force beneath the model. Second, a frozen model cannot interpret raw force streams, so force must be returned in a form the model can reason about. Therefore, we present ForceHarness, a harness that lets frozen models use force without demonstrations or finetuning. After calibrating hand force from a wrist sensor or joint torques alone, ForceHarness addresses the first challenge with self-terminating force primitives that regulate force on their own and return at a contact event. To address the second challenge, a physics translation layer turns each force trace into physical quantities with uncertainty as the tool result. To evaluate ForceHarness against expert force controllers and learned policies under identical conditions, we also build HiddenPhys, a seven-task MuJoCo benchmark with hidden physical variables. On a real arm without a wrist force sensor, ForceHarness raises the zero-shot success rate of GPT-6 Astra from 3.3% to 90.0%. In simulation, it adds 17.5 percentage points on the public ForceBench suite and 34.3 percentage points on HiddenPhys, surpassing force-aware policies trained on demonstrations on both benchmarks and matching hand-written expert controllers on HiddenPhys. Ablations further show that smaller models make significantly better decisions when given physical quantities rather than raw force values.
Community
Hi everyone! 👋 Excited to share ForceHarness, an agent harness that lets frozen API frontier models understand and use wrist force for contact-rich manipulation—with zero demonstrations and zero finetuning.
Why ForceHarness?
Frontier models can control robots zero-shot from images, yet cameras cannot see contact forces (whether a USB plug has seated, whether a screw thread caught, which bottle is heavier, or whether a door is locked vs. merely stiff). Meanwhile, contact evolves far faster than an API call, and frozen models cannot interpret raw force streams.
ForceHarness bridges this gap with two layers:
- Self-terminating force primitives close the fast force loop beneath the model (
search,wiggle,screw,slide,lift, ...) and stop automatically on contact events while holding safely as the model thinks. - Physics translation layer converts each force trace into physical quantities with uncertainty (mass, friction, stiffness, contact height, insertion advance)—with no hardcoded task thresholds.
Key Highlights
- 3.3% → 90.0% zero-shot on a real Franka FR3 (no wrist F/T sensor): Across 6 contact-rich tasks (USB insertion, screw driving, eggplant peeling, whiteboard wiping, bottle weight sorting, and firmness probing) using joint torques alone.
- Surpasses trained force-aware VLAs (200 demos/task): Beats $\pi_{0.5}$, ForceVLA, and Diffusion Policy on both ForceBench (82.5% held-out vs. 35.3% best trained) and HiddenPhys, our new 7-task MuJoCo hidden-physics benchmark (96.4% / 135/140, matching expert force controllers with 0 damaged handles).
- Drop-in Agent Skill (
SKILL.md): Give the repo to your coding agent (Claude Code, Codex, Cursor, etc.) and it will check force sensing capability, calibrate, and bring up your simulator or real robot arm gate by gate—never moving hardware without your confirmation.
Links
- 🌐 Project Page & Videos: https://showlab.github.io/ForceHarness/
- 💻 Code & Agent Setup Skill: https://github.com/showlab/ForceHarness
- 📊 HiddenPhys Dataset (1,423 expert demos, raw + LeRobot): https://huggingface.co/datasets/Jack97/ForceHarness-Data
Happy to answer any questions or hear your feedback!
Interesting paper! I found your paper is also related to our recent work (PEARS: Physical-Prior-Guided Efficient Adaptation via Failure Reasoning and Diffusion Steering for Tactile Manipulation, https://arxiv.org/abs/2610.08784). It will be great if you can provide a short discussion in your work in the future version.
Get this paper in your agent:
hf papers read 2611.21099 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper