Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Test
Published in arXiv preprint (under review), 2026
Overview
Before using video foundation models for downstream embodied tasks, two critical questions remain: (1) Is their generative generalization sufficient for human observers? (2) Are they robust enough as universal priors for real-world embodied agents?
Key Contributions
Embodied Turing Test: WoW-World-Eval provides a standardized framework for evaluating world models in embodied AI.
Five Core Abilities: Examines perception, planning, prediction, generalization, and execution using 609 robot manipulation scenarios.
22 Evaluation Metrics: Comprehensive protocol achieving >0.93 Pearson correlation with human preference.
Key Findings: Models achieve only 17.27 on long-horizon planning and 68.02 on physical consistency, revealing gaps between generated videos and real-world physics.
IDM Turing Test: Most models collapse to ~0% success in real-world execution, while WoW maintains 40.74% success rate.
Recommended citation: Chun-Kai Fan*, Xiaowei Chi*, ... Zhiyuan Jiang, ... et al. (2026). "Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Test." arXiv preprint arXiv:2601.04137.
Download Paper