DominoBench: Evaluating Multi-Step Physical Reasoning from Still Images
Abstract
A new benchmark, DominoBench, shows that current vision-language and multimodal models cannot reliably predict physical outcomes from still images unless the critical calculation and necessary quantities are provided.
Can a vision-language model look at a picture of a chain reaction and tell what will happen? Recent multimodal models describe scenes fluently and solve textbook physics problems, which suggests they should also predict how a scene of ramps, ledges, pendulums, seesaws and dominoes plays out once a ball is pushed. We tested this claim on 30 models from 10 families and reached a clear conclusion: unaided, no model predicts the outcome reliably. We built DominoBench, a benchmark of still images of 2D rigid-body scenes whose outcome a physics simulator computes. It holds 854 items in 427 minimal pairs, in which a change of one to three grid steps in one quantity decides whether one person survives, and every item comes with a four-rung hint ladder ending in the exact calculation that decides the outcome. Unaided, frontier models from Anthropic, Google and OpenAI are right about the person whose fate flips in at most 72% of items and answer both twins of a pair correctly in at most 34%. Pointing at the deciding step helps little, and exact geometry given as text barely helps, yet once the deciding calculation and the quantities it needs, including speeds that a still image cannot show, are written out, the same models reach 82-92% on the deciding person, while small open-weight models stay near chance. Physical prediction from a still image therefore fails at the quantitative step: the strongest models use the deciding quantity when they are given it, but do not produce it on their own. For models that plan or act in the physical world, a fluent account of a scene is thus no evidence of a correct prediction: such models must find and compute the deciding quantity unprompted, and DominoBench tests whether they do.
Community
Can a vision-language model look at a still image of a chain reaction and tell who gets hit? DominoBench tests exactly that: 854 simulator-labelled scenes in minimal pairs, where one quantity moves one grid step and one person's fate flips.
What I found across 30 models:
- Unaided, no model is reliable. The best are right about the person whose fate flips in at most 75% of items, and get both twins of a pair right in at most 40%.
- Pointing at the deciding step helps little. Writing out the calculation helps a lot: the same models reach 80โ92%.
- Two families behave very differently. The GPT-6 models are the most accurate unaided but barely gain from the calculation, while GPT-5.6-Sol starts low and ends first.
Everything is open: images, simulator trajectories, the hint ladder, the generator and the evaluation code.
Dataset: https://huggingface.co/datasets/emilioferrara/DominoBench
I'd love to hear whether your models do better, and which mechanism trips them up.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- See, Think, Use: Diagnosing Perception and Reasoning Failures in VLMs (2026)
- PRISM-BENCH: Diagnosing Grounded and Faithful Multimodal Reasoning Beyond Final-Answer Accuracy (2026)
- DecoyCount: Evaluating Visual-Reasoning Misalignment in Counting (2026)
- Asking What a Director Sees Can Change What Vision-Language Models Choose (2026)
- Geometric Parrots: Rule Absence vs. Rule Fragility in Spatial and Physical Reasoning for Vision-Language Models (2026)
- S-Agent 2.0: Spatial Reasoning with a General Coding Agent (2026)
- Give the Model a Finger: Developmentally Aligned Evaluation of Counting in Vision-Language Models (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2611.20385 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper