Zoom in, Click out: Unlocking and Evaluating the Potential of Zooming for GUI Grounding
Published in arXiv preprint (under review), 2025
Overview
GUI grounding—the ability to locate UI elements from natural language—is fundamental for building autonomous GUI agents. Existing approaches rely on large-scale bounding box supervision but still struggle with cross-platform generalization, complex layouts, and fine-grained localization.
Key Contributions
Zoom as a Prior: We discover that zooming provides a powerful yet underexplored prior for GUI grounding, enabling dynamic spatial focusing and adaptive context switching.
Training-Free Method: ZoomClick requires no additional training and works with both general VLMs and specialized GUI grounding models.
Four Key Properties: We characterize pre-zoom, depth, shrink size, and minimal crop size to unlock zoom’s full potential.
State-of-the-Art Results: UI-Venus-72B achieves 73.1% success rate on ScreenSpot-Pro; smaller models match larger ones’ performance.
GUIZoom-Bench: A new benchmark for evaluating model adaptability to zoom, enabling future research on test-time scaling.
| arXiv | Code |
Recommended citation: Zhiyuan Jiang, Shenghao Xie, Wenyi Li, Wenqiang Zu, Peihang Li, Jiahao Qiu, Siqi Pei, Lei Ma, Tiejun Huang, Mengdi Wang, Shilong Liu. (2025). "Zoom in, Click out: Unlocking and Evaluating the Potential of Zooming for GUI Grounding." arXiv preprint arXiv:2512.05941.
Download Paper