Post
16
One frozen encoder now maps 3 tactile sensor types (piezoresistive, magnetic, visuotactile) into the same token space, and the Tiny checkpoint runs on CPU.
LingBot-Touch bets that touch progresses by standardizing the representation, not by building a better sensor. A JEPA-style model uses modality-specific tokenizers and one shared Transformer, predicting masked tactile content in latent space. The authors say the tokens plug into diffusion policy and VLAs such as pi0.5 and LingBot-VLA. The paper has 91 upvotes on HF Daily Papers.
🔍 What's released
• Frozen target encoder plus utilities that output tokens [B,N,C] with time, finger and position addresses
• Tiny and Small checkpoints, Apache-2.0, with 120 sample files for Sharpa and XHand
• Inference only: pretraining code and policy integrations are not described as released
🛠️ Try it
• Run infer.py on the Sharpa or XHand sample with --device cpu
• Read out with cross-attention queries, not mean/max pooling: each token mixes information from other taxels and time steps
⚠️ Caveats
• A new raw taxel layout needs verified geometry calibration; matching tensor shapes is not enough
• The abstract gives no per-task numbers, so check the paper's tables before trusting the gains
• Model files need an account with access to the model repo
Learning Heterogeneous Tactile Representation for General Dexterous Manipulation (2611.12806)
https://github.com/Robbyant/lingbot-touch
Would you put a frozen cross-sensor tactile encoder in front of your policy, or keep per-sensor pipelines?
📚 Read it elsewhere
• The deep dive on Medium: https://medium.com/p/0d2028d8fc73
• The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/lingbot-touch-argues-touch-needs
• The thread on X: https://x.com/saavoji/status/2120209644392194083
• Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#lingbot-touch
LingBot-Touch bets that touch progresses by standardizing the representation, not by building a better sensor. A JEPA-style model uses modality-specific tokenizers and one shared Transformer, predicting masked tactile content in latent space. The authors say the tokens plug into diffusion policy and VLAs such as pi0.5 and LingBot-VLA. The paper has 91 upvotes on HF Daily Papers.
🔍 What's released
• Frozen target encoder plus utilities that output tokens [B,N,C] with time, finger and position addresses
• Tiny and Small checkpoints, Apache-2.0, with 120 sample files for Sharpa and XHand
• Inference only: pretraining code and policy integrations are not described as released
🛠️ Try it
• Run infer.py on the Sharpa or XHand sample with --device cpu
• Read out with cross-attention queries, not mean/max pooling: each token mixes information from other taxels and time steps
⚠️ Caveats
• A new raw taxel layout needs verified geometry calibration; matching tensor shapes is not enough
• The abstract gives no per-task numbers, so check the paper's tables before trusting the gains
• Model files need an account with access to the model repo
Learning Heterogeneous Tactile Representation for General Dexterous Manipulation (2611.12806)
https://github.com/Robbyant/lingbot-touch
Would you put a frozen cross-sensor tactile encoder in front of your policy, or keep per-sensor pipelines?
📚 Read it elsewhere
• The deep dive on Medium: https://medium.com/p/0d2028d8fc73
• The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/lingbot-touch-argues-touch-needs
• The thread on X: https://x.com/saavoji/status/2120209644392194083
• Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#lingbot-touch
