Why it matters
Existing benchmarks for physics reasoning in foundation models often rely on synthetic scenes and high-level event analysis, which may not accurately assess true low-level Newtonian understanding. NewtPhys addresses this gap by offering a dataset with high visual fidelity and fine-grained physical annotations, enabling more rigorous evaluation and fostering the development of physics-aware AI models. This could lead to more robust and reliable AI systems in applications requiring a deep understanding of physical interactions.

A new research paper introduces NewtPhys, a novel 4D physically annotated dataset aimed at assessing how well foundation models comprehend Newtonian physics. Unlike previous benchmarks that often use synthetic or semi-synthetic scenes and focus on high-level events, NewtPhys is constructed from multiview images of real-world scenes, augmented with physics-grounded simulations. This approach provides dense, fine-grained annotations across timesteps, encompassing 3D forces and amodal per-pixel quantities related to physics, tracking, semantics, and geometry.

The creators of NewtPhys utilized this dataset to conduct a systematic evaluation of 56 Vision-Language Models (VLMs), including 54 open-weight and 2 closed-source models, alongside 10 Vision Foundation Models (VFMs). The findings indicate that these models exhibit limitations in their low-level physics reasoning abilities. The researchers suggest that NewtPhys can serve as a valuable resource for future research in physics-grounded vision and for developing advanced physics-aware evaluation methods. Code and datasets are publicly available.

Sources

Written with AI assistance from the linked sources; every claim below was checked against them automatically. How we produce articles.

Claim check: 4/4 supported claims - 4 evidence links - 89% avg confidence
  • Researchers introduced NewtPhys, a 4D physically annotated dataset for evaluating foundation models' understanding of Newtonian physics.supported - arxiv.org
  • NewtPhys is built from multiview images of real-world scenes with physics-grounded simulations and provides dense, fine-grained annotations including 3D forces and per-pixel quantities.supported - arxiv.org
  • Evaluations using NewtPhys on 56 VLMs (54 open-weight, 2 closed-source) and 10 VFMs revealed limitations in their low-level physics reasoning.supported - arxiv.org
  • Code and datasets for NewtPhys are available at https://astra-vision.github.io/NewtPhys.supported - arxiv.org

Caveats

  • Single-source caution: verify critical details at the linked source.
Radar score 82/100 - how it was calculated
Reliability80
Freshness90
Novelty87
Technical90
Developer68
Ecosystem68
Confidence96
  • Reliability 80: Research metadata source
  • Freshness 90: Fresh research date
  • Novelty 87: Research implementation signal
  • Technical 90: Research technical evidence
  • Developer 68: Research developer relevance
  • Ecosystem 68: Research implementation signal
  • Confidence 96: Claims have reliable evidence
physicsbenchmarkingfoundation modelsvision-language modelsvision foundation modelsdataset
Share
XLinkedInHacker News

Discussion

Loading comments...

Related articles

Research Papers - Sep 19, 2026SplashSplat: Reconstructing Splashing Liquids from Real-World Multi-View VideosResearchers have introduced SplashSplat, a novel method for reconstructing splashing liquids from real-world multi-view videos. They also present a new benchmark dataset of 20 real-world scenes, captured with synchronized 4K cameras at 60 fps.Research Papers - Sep 18, 2026PANORAMA: Panoptic Grounded Captioning via Mask Proposal SelectionResearchers have introduced PANORAMA, a vision-language model designed for panoptic grounded captioning, which aims to generate detailed scene descriptions with precise pixel-level grounding. The model is accompanied by PanoCaps, a new human-annotated benchmark dataset for training and evaluating this task.Research Papers - Sep 13, 2026AmazonWSE Dataset and Model for River Water Elevation ImputationResearchers have introduced AmazonWSE, a new dataset and a sequence-based model for imputing water surface elevation (WSE) in the Amazon river basin. The dataset addresses the challenge of extreme data sparsity from satellite altimetry and in-situ gauges, covering over 19,000 river sections across a decade.Research Papers - Sep 28, 2026Ego-Exo4D-HM: 4D Human Motion Reconstruction Dataset and PipelineResearchers have introduced Ego-Exo4D-HM, a large-scale dataset featuring 4D human motion reconstructions derived from the Ego-Exo4D dataset's synchronized egocentric and multi-view exocentric video captures. This release also includes the accompanying reconstruction pipeline, which adapts state-of-the-art methods to leverage the multi-camera setup for improved accuracy.Research Papers - Sep 29, 2026PoEM: Predicting RL Outcomes from Existing PoliciesResearchers have introduced PoEM, a framework designed to predict the outcomes of reinforcement learning (RL) on foundation models without requiring new RL training. This approach leverages existing post-trained models to estimate new policies based on new reward functions.Research Papers - Sep 27, 2026New Benchmark for Evaluating LLMs in EHR Information RetrievalResearchers have developed the Benchmark for Retrieving Information in EHRs (BRIE), a scalable framework that automatically generates question-answer pairs from longitudinal EHR notes. This "living" benchmark aims to provide continuous, up-to-date evaluation of clinical LLMs, addressing limitations of static, manually curated datasets.