Model Comparison
Hands squeeze an empty plastic water bottle, place the opening over an egg yolk in a bowl, and release the squeeze, sucking the yolk into the bottle, leaving the white behind.
Tested concepts:
fundamental physics (
),
object properties & affordances (
),
spatial reasoning (
),
action & procedural understanding (
),
force and motion (
)
Text-to-video (T2V) models now produce striking visuals, yet they routinely violate everyday physics; objects float, tools are misused, and causal sequences break down. Existing benchmarks mostly probe isolated physical laws and rely on vision-language models to score videos directly, which entangles perception and reasoning in one judgment and correlates poorly with humans. We introduce PhysVidBench, a benchmark of 383 base prompts, expanded to 766 prompts with enriched variants and 4,123 manually reviewed QA items, together with a language-grounded evaluation framework that takes a different route: instead of asking a VLM “Is this video physically correct?”, we caption the video, then ask a language model to answer prompt-derived yes/no physics questions using only the captions. This split between seeing and reasoning makes each score traceable to the captions that justify it and aligns more closely with human judgment than direct VLM scoring (Pearson r = 0.45–0.69, compared with 0.21–0.47 for the strongest direct VLM evaluator). To test the framework, we carefully curate a set of human-validated prompts spanning seven physical dimensions, with a focus on tool use and affordances – areas largely absent from prior physics-focused benchmarks. Across 12 open and proprietary T2V systems, the best model reaches only 36.2% average accuracy, and no model consistently handles everyday physical reasoning. The same pipeline can also guide iterative error-guided prompt refinement, improving CogVideoX-2B from 21.6 to 32.7 and CogVideoX-5B from 17.8 to 29.7 without retraining.
Physical commonsense dimensions evaluated in PhysVidBench, illustrated through video examples generated by diverse text-to-video (T2V) models. The seven dimensions are multi-label rather than mutually exclusive: a prompt may involve multiple aspects of physical reasoning, including force and motion, object properties and affordances, spatial reasoning, temporal dynamics, and material interaction and transformation.
Video Generation Pipeline. An overview of the four-stage pipeline used to construct PhysVidBench. We begin with correct solutions from the PIQA dataset and filter them using a large language model (LLM) to identify instances of secondary tool use and object affordances (Stage 1). These filtered solutions are converted into short, physically grounded video generation prompts via another LLM pass (Stage 2). Each base prompt is further upsampled to enhance its physical specificity and causal structure while preserving the original scene (Stage 3). Both base and upsampled prompts are fed into a diverse set of state-of-the-art text-to-video generation models, resulting in videos that probe tool interaction and physical commonsense (Stage 4).
Evaluation Pipeline. Overview of our three-stage evaluation framework designed to assess physical commonsense in generated videos. Using the upsampled prompts, we generate yes/no questions aligned with one or more of the seven physical commonsense dimensions in our ontology (Stage 1). Each generated video is captioned using AuroraCap, which produces a general-purpose and seven dimension-specific captions aimed at surfacing evidence relevant to different types of physical reasoning (Stage 2). The LLM is prompted to answer the targeted physics question using only the video captions (Stage 3).
| Model | AU |
FM |
FP |
MT |
OP |
SR |
TD |
Avg |
|---|---|---|---|---|---|---|---|---|
| LTX-Video | 20.6 (-4.2) | 19.7 (-3.4) | 18.0 (-2.1) | 16.4 (-1.5) | 24.3 (-4.4) | 17.1 (-2.4) | 14.1 (-4.8) | 20.7 (-2.8) |
| VideoCrafter2 | 17.7 (-1.6) | 19.3 (-2.8) | 22.6 (-2.8) | 18.3 (-2.8) | 26.4 (-2.6) | 15.7 (-0.2) | 12.3 (-1.9) | 22.0 (-1.8) |
| CogVideoX (2B) | 22.2 (-1.5) | 24.4 (-4.3) | 23.3 (-2.8) | 22.5 (-0.4) | 29.2 (-4.0) | 19.5 (-1.3) | 16.7 (-1.5) | 25.6 (-3.0) |
| CogVideoX (5B) | 18.7 (-0.8) | 19.4 (-3.1) | 20.7 (-3.2) | 17.8 (-1.5) | 25.6 (-1.8) | 17.9 (-0.4) | 12.6 (+0.8) | 20.8 (-1.5) |
| Wan2.1 (1.3B) | 30.0 (-3.6) | 28.9 (-5.3) | 28.2 (-3.3) | 28.5 (-2.0) | 35.1 (-3.9) | 27.3 (-3.3) | 19.0 (-0.0) | 30.5 (-3.2) |
| Wan2.1 (14B) | 33.3 (-3.4) | 31.4 (-1.7) | 31.9 (-1.4) | 32.4 (-3.3) | 39.1 (-4.5) | 29.8 (-1.5) | 23.4 (-1.5) | 33.9 (-1.9) |
| MAGI-1 | 27.2 (-6.0) | 26.3 (-3.8) | 28.8 (-1.6) | 27.7 (-4.6) | 37.3 (-5.7) | 30.3 (-5.3) | 19.3 (-8.1) | 32.5 (-5.5) |
| HunyuanVideo | 26.4 (-0.6) | 25.8 (+0.5) | 27.7 (-0.1) | 31.9 (-4.8) | 35.7 (-1.9) | 23.9 (+2.2) | 21.6 (-5.2) | 30.1 (-1.2) |
| Cosmos (7B) | 32.1 (-8.1) | 32.4 (-9.1) | 32.9 (-8.6) | 35.8 (-11.0) | 39.6 (-10.5) | 31.7 (-10.1) | 21.6 (-8.2) | 34.8 (-8.3) |
| Cosmos (14B) | 35.7 (-6.0) | 33.3 (-6.1) | 30.9 (-4.0) | 36.8 (-6.0) | 41.0 (-9.8) | 32.7 (-5.6) | 20.8 (-0.3) | 36.1 (-7.1) |
| Sora | 28.6 (+2.4) | 30.0 (-1.1) | 29.9 (+0.2) | 33.1 (+0.3) | 37.0 (-1.1) | 24.8 (+1.7) | 16.7 (+2.6) | 31.4 (+0.3) |
| Veo-2 | 34.9 (-0.7) | 33.8 (-2.3) | 31.7 (-2.7) | 36.0 (-3.1) | 38.4 (-1.5) | 29.7 (+1.0) | 19.5 (+2.5) | 34.8 (-0.7) |
AU: Action & Procedural Understanding, FM: Force and Motion, FP: Fundamental Physics, MT: Material Interaction & Transformation, OP: Object Properties & Affordances, SR: Spatial Reasoning, TD: Temporal Dynamics.
We present evaluation results for 12 state-of-the-art video generation models using PhysVidBench, which measures physical commonsense across seven fine-grained dimensions: Action & Procedural Understanding, Force & Motion, Fundamental Physics, Material Interaction & Transformation, Object Properties & Affordances, Spatial Reasoning, and Temporal Dynamics, along with an overall average. Each model is tested under two prompt variants: base prompts derived from PIQA scenarios and upsampled prompts that enrich physical details and causal structure. Prompt upsampling generally improves performance, with gains varying across models. This suggests that providing more explicit physical context can help elicit stronger physical behavior from generative models. Despite these gains, Spatial Reasoning and Temporal Dynamics remain among the most challenging dimensions, reflecting persistent difficulties in preserving geometric relations, event order, and temporal causality. Interestingly, while larger models such as Wan2.1-14B and Cosmos-14B generally achieve higher scores, size alone does not guarantee robust physical understanding, suggesting that factors beyond parameter count also play an important role. Overall, PhysVidBench provides a clear and interpretable framework for diagnosing such gaps and benchmarking progress in physically grounded video generation.
