The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space

Brian Potetz; Chun-Ta Lu; Howard Zhou; Leonidas Guibas; Xia Hu; Zhenrui Yue; Zhicheng Wang

arxiv: 2605.09883 · v2 · pith:LG5OKHXJnew · submitted 2026-05-11 · 💻 cs.CV · cs.AI

The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space

Xia Hu , Zhenrui Yue , Brian Potetz , Howard Zhou , Leonidas Guibas , Chun-Ta Lu , Zhicheng Wang This is my paper

Pith reviewed 2026-06-30 22:55 UTC · model grok-4.3

classification 💻 cs.CV cs.AI

keywords visual reasoningmultimodal large language modelsCartesian shortcutpolar coordinatesbenchmarksspatial topologycoordinate invariance

0 comments

The pith

Models achieve high visual reasoning scores by discretizing grid images into text but collapse when the same tasks use polar coordinates.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that current multimodal models exploit an orthogonal grid prior in benchmarks, converting layouts into explicit textual coordinates for deductive reasoning instead of performing visual analysis. To test this, the authors create paired Cartesian and polar versions of 53 tasks that keep identical logic and semantics. Frontier models that reach 70-83 percent accuracy on the Cartesian versions fall to 31-39 percent on the polar versions, and any reasoning gains seen on grids largely disappear. The result indicates that these models lack visual reasoning that remains stable across different spatial topologies.

Core claim

The Cartesian Shortcut allows models to bypass genuine visual processing by turning grid-based images into discrete text coordinates; when tasks are instead expressed in polar space while preserving every logical constraint, performance of leading MLLMs drops sharply and stays low, showing that their reasoning is not invariant to coordinate topology.

What carries the argument

Polaris-Bench, a collection of 53 visual reasoning tasks reformulated in polar coordinate space together with their Cartesian counterparts, engineered to eliminate the orthogonal grid structure that models exploit.

If this is right

High scores on standard Cartesian benchmarks do not demonstrate robust visual understanding.
Reasoning improvements measured on grid-based tasks largely fail to transfer when the spatial representation changes.
Current models cannot reliably solve the same logical problems once the underlying topology is altered.
Development of topology-invariant visual reasoning is required before benchmark saturation can be treated as genuine progress.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same shortcut could appear in other non-grid representations such as spherical or cylindrical coordinates used in 3D or panoramic data.
Applications that naturally produce radial or angular observations, including certain robotics or medical imaging settings, may reveal comparable weaknesses.
Augmenting training data with polar-transformed versions of existing tasks could encourage more invariant internal representations.

Load-bearing premise

Converting the original tasks into polar coordinates keeps the logical constraints and task meaning exactly the same and adds no extra visual or reasoning difficulty.

What would settle it

A controlled test in which models reach similar accuracy on the polar and Cartesian versions even when image-to-text coordinate conversion is blocked would falsify the shortcut claim.

read the original abstract

As current Multimodal Large Language Models rapidly saturate canonical visual reasoning benchmarks, a key question emerges: do these strong scores genuinely reflect robust visual understanding? We identify a pervasive vulnerability, the Cartesian Shortcut: visual reasoning benchmarks prevalently build on orthogonal grid-based layouts that can be readily discretized into explicit textual coordinates. Models systematically exploit this property, heavily leveraging text-based deductive reasoning to assist visual problem-solving. To systematically dismantle this shortcut, we introduce Polaris-Bench, which re-formulates 53 visual reasoning tasks in Polar coordinate space with paired Cartesian counterparts as reference, while preserving consistent logical constraints and task semantics -- thus fundamentally breaking the orthogonal prior that models exploit. Comprehensive evaluation across $14$ state-of-the-art MLLMs reveals that frontier models achieving $70$--$83\%$ on Cartesian layouts collapse to $31$--$39\%$ on Polar equivalents, with degradation persisting even under complete logical equivalence. Moreover, reasoning gains observed on Cartesian layouts are severely diminished on Polar equivalents. These findings expose a critical deficiency in current MLLMs: the lack of topology-invariant visual reasoning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The performance collapse on polar reformulations is the main result, but the paper still needs to show those versions are truly equivalent in difficulty.

read the letter

The paper's key observation is that 14 MLLMs drop from 70-83% accuracy on standard visual reasoning tasks to 31-39% when the same 53 tasks are rewritten in polar coordinates. The authors tie this to models exploiting the orthogonal grid structure that Cartesian layouts make easy to turn into text coordinates.

Polaris-Bench itself is the concrete new piece. Creating paired Cartesian and polar versions of the tasks gives a direct way to measure how much performance depends on that grid property. The evaluation is broad enough to show the pattern holds across frontier models and that reasoning gains seen on Cartesian versions largely vanish on the polar ones.

The soft spot is the claim that the polar versions preserve identical logical constraints and task semantics. Shifting to polar coordinates changes how distances, angles, and alignments are rendered in the image. That shift can add demands around curvature, discretization, or visual parsing that Cartesian grids avoid. The abstract states the versions are equivalent but gives no human accuracy parity check, no ablation on transformation complexity, and no other control that would isolate the shortcut as the only cause. Without those, part of the drop could come from the reformulation itself.

This work is for people who build or evaluate multimodal models and care about whether benchmarks actually test visual reasoning. Anyone running robustness tests or designing new visual tasks would get direct use from the paired dataset.

It deserves peer review. The empirical pattern is new and the question matters for how we measure MLLM capabilities, even if the current version needs added controls to pin down the cause.

Referee Report

2 major / 1 minor

Summary. The paper claims that MLLMs exploit a pervasive 'Cartesian Shortcut' in visual reasoning benchmarks that rely on orthogonal grid layouts, allowing models to discretize inputs into explicit textual coordinates and bypass robust visual understanding. To expose this, the authors introduce Polaris-Bench, which reformulates 53 tasks into polar coordinate space (with paired Cartesian references) while asserting preservation of logical constraints and task semantics; evaluation of 14 frontier MLLMs shows accuracy collapsing from 70-83% on Cartesian versions to 31-39% on polar equivalents, with diminished reasoning gains, indicating a lack of topology-invariant visual reasoning.

Significance. If the equivalence of logical constraints holds, the result would be significant for the field: it would demonstrate that high benchmark scores on canonical visual reasoning tasks do not reflect genuine visual or topological understanding, but rather exploitation of grid-based priors. The scale of the evaluation (14 models, 53 tasks) and the introduction of a new benchmark with explicit Cartesian-polar pairs would provide a concrete, falsifiable test of shortcut reliance, potentially influencing future benchmark design and training objectives.

major comments (2)

[Abstract] Abstract: the central claim that performance collapse is attributable to removal of the orthogonal shortcut rests on the assertion that polar reformulations 'preserve consistent logical constraints and task semantics'; however, no quantitative controls (human accuracy parity, ablation on transformation complexity, or discretization effects) are referenced to rule out new perceptual or representational difficulties introduced by polar coordinates.
[Abstract] The abstract reports clear performance drops across 14 models but supplies no implementation details, verification of logical equivalence between paired tasks, error bars, or controls for task difficulty; this information is load-bearing for interpreting whether the 31-39% polar scores reflect the claimed deficiency rather than uncontrolled side effects of the coordinate transformation.

minor comments (1)

The invented term 'Cartesian Shortcut' is used without an early formal definition or pseudocode for how the shortcut is operationalized in the 53 tasks.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive feedback. We address the two major comments on the abstract below, clarifying the basis for our claims while committing to revisions that add requested controls and details.

read point-by-point responses

Referee: [Abstract] Abstract: the central claim that performance collapse is attributable to removal of the orthogonal shortcut rests on the assertion that polar reformulations 'preserve consistent logical constraints and task semantics'; however, no quantitative controls (human accuracy parity, ablation on transformation complexity, or discretization effects) are referenced to rule out new perceptual or representational difficulties introduced by polar coordinates.

Authors: Logical equivalence is maintained by direct, rule-preserving reformulation: each polar task is constructed from its Cartesian pair by remapping only the coordinate representation while keeping identical logical constraints, objectives, and solution paths (detailed with examples in Section 3). We agree that explicit quantitative validation would further rule out transformation artifacts. In revision we will add (i) human accuracy parity results on a 10-task subset and (ii) an ablation examining discretization granularity and transformation complexity. revision: yes
Referee: [Abstract] The abstract reports clear performance drops across 14 models but supplies no implementation details, verification of logical equivalence between paired tasks, error bars, or controls for task difficulty; this information is load-bearing for interpreting whether the 31-39% polar scores reflect the claimed deficiency rather than uncontrolled side effects of the coordinate transformation.

Authors: Space limits prevent the abstract from containing these elements; they appear in the full manuscript (model implementation and evaluation protocol in Section 4; paired-task equivalence verification and difficulty matching via Cartesian-polar design in Section 3). We will incorporate error bars on all reported accuracies and add an explicit discussion of potential side effects in the revised abstract and main text. revision: partial

Circularity Check

0 steps flagged

No significant circularity; purely empirical benchmark comparison

full rationale

The paper's core contribution is the creation of Polaris-Bench via reformulation of 53 tasks into polar coordinates (with Cartesian pairs) and subsequent empirical evaluation of 14 MLLMs, showing performance collapse from 70-83% to 31-39%. No mathematical derivations, parameter fittings, predictions derived from fits, or load-bearing self-citations are present. The preservation of logical constraints is an assumption of the methodology rather than a derived result, and the findings are externally falsifiable via model testing on the released benchmark. This matches the default expectation of non-circular empirical work.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

This is an empirical benchmark paper; the primary unverified premise is task equivalence under coordinate change, with the Cartesian Shortcut serving as a descriptive label rather than a derived entity.

axioms (1)

domain assumption The polar reformulation maintains identical logical constraints and task semantics to the Cartesian version.
Stated directly in the abstract as the foundation for the comparison.

invented entities (1)

Cartesian Shortcut no independent evidence
purpose: To name the observed model behavior of exploiting textual coordinates in grid-based benchmarks.
Conceptual label introduced by the authors to describe the vulnerability; no independent evidence provided.

pith-pipeline@v0.9.1-grok · 5736 in / 1223 out tokens · 29201 ms · 2026-06-30T22:55:46.332649+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Reference graph

Works this paper leans on

50 extracted references · 16 canonical work pages · 7 internal anchors

[1]

M. AI. Mistral-small-3.1-24b-instruct-2503, March 2025. URLhttps://mistral.ai/news/ mistral-small-3-1

2025
[2]

Anthropic

A. Anthropic. The claude 3 model family: Opus, sonnet, haiku.Claude-3 Model Card, 1(1):4, 2024

2024
[3]

G. B. Arfken, H. J. Weber, and F. E. Harris.Mathematical methods for physicists: a comprehensive guide. Academic press, 2011

2011
[4]

Asadi, J

M. Asadi, J. W. O’Sullivan, F. Cao, T. Nedaee, K. Rajabalifardi, F.-F. Li, E. Adeli, and E. Ashley. Mirage: The illusion of visual understanding, 2026. URLhttps://arxiv.org/abs/2603. 21687

2026
[5]

M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges.arXiv preprint arXiv:2104.13478, 2021

work page internal anchor Pith review Pith/arXiv arXiv 2021
[6]

J. Chen, T. Liang, S. Siu, Z. Wang, K. Wang, Y. Wang, Y. Ni, W. Zhu, Z. Jiang, B. Lyu, D. Jiang, X. He, Y. Liu, H. Hu, X. Yue, and W. Chen. Mega-bench: Scaling multimodal evaluation to over 500 real-world tasks. 2025

2025
[7]

L. Chen, J. Li, X. Dong, P. Zhang, Y. Zang, Z. Chen, H. Duan, J. Wang, Y. Qiao, D. Lin, et al. Are we on the right way for evaluating large vision-language models?Advances in Neural Information Processing Systems, 37:27056–27087, 2024

2024
[8]

L. Chen, W. Xie, Y. Liang, H. He, H. Zhao, Z. Yang, Z. Huang, H. Wu, H. Lu, Y. charles, Y. Bao, Y. Fan, G. Li, H. Shen, X. Chen, W. Xu, S. Si, Z. Cai, W. Chai, Z. Huang, F. Liu, T. Liu, B. Chang, X. Hu, K. Chen, Y. Ren, Y. Liu, Y. Gong, and K. Li. Babyvision: Visual reasoning beyond language,
[9]

URLhttps://arxiv.org/abs/2601.06521

work page arXiv
[10]

Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities

G.Comanici, E.Bieber, M.Schaekermann, I.Pasupat, N.Sachdeva, I.Dhillon, M.Blistein, O.Ram, D. Zhang, E. Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodal- ity, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025
[11]

DeepMind

G. DeepMind. Gemma 4: Our most intelligent open models to date. https://blog. google/innovation-and-ai/technology/developers-tools/gemma-4/, 2026. Ac- cessed: 2026-05-01

2026
[12]

Q. Dong, L. Li, D. Dai, C. Zheng, J. Ma, R. Li, H. Xia, J. Xu, Z. Wu, T. Liu, B. Chang, X. Sun, L. Li, and Z. Sui. A survey on in-context learning, 2024. URLhttps://arxiv.org/abs/2301. 00234

2024
[13]

BLINK: Multimodal Large Language Models Can See but Not Perceive

X.Fu, Y.Hu, B.Li, Y.Feng, H.Wang, X.Lin, D.Roth, N.A.Smith, W.-C.Ma, andR.Krishna. Blink: Multimodal large language models can see but not perceive.arXiv preprint arXiv:2404.12390, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024
[14]

Geirhos, J.-H

R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann. Shortcut learning in deep neural networks.Nature Machine Intelligence, 2(11):665–673, 2020

2020
[15]

Gemini 3 flash: Frontier intelligence built for speed

Gemini Team and Google DeepMind. Gemini 3 flash: Frontier intelligence built for speed. Technical blog / model release, Google, 12 2025. URL https://blog.google/ products-and-platforms/products/gemini/gemini-3-flash/. 12 The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space

2025
[16]

Gemini 3.1 flash-lite: Built for intelli- gence at scale

Gemini Team and Google DeepMind. Gemini 3.1 flash-lite: Built for intelli- gence at scale. Technical blog / model release, Google, 3 2026. URL https: //blog.google/innovation-and-ai/models-and-research/gemini-models/ gemini-3-1-flash-lite/

2026
[17]

Gemini 3 pro technical report

Gemini Team, Google. Gemini 3 pro technical report. 2025. URL https://deepmind. google/models/gemini/pro/

2025
[18]

J. Gu, X. Jiang, Z. Shi, H. Tan, X. Zhai, C. Xu, W. Li, Y. Shen, S. Ma, H. Liu, et al. A survey on llm-as-a-judge.The Innovation, 2024

2024
[19]

Y. Hao, J. Gu, H. W. Wang, L. Li, Z. Yang, L. Wang, and Y. Cheng. Can MLLMs reason in multimodality? EMMA: An enhanced multimodal reasoning benchmark. InForty-second In- ternational Conference on Machine Learning, 2025. URLhttps://openreview.net/forum? id=v26vwjxOEz

2025
[20]

M. Jia, Z. Qi, S. Zhang, W. Zhang, X. Yu, J. He, H. Wang, and L. Yi. Omnispatial: Towards comprehensive spatial reasoning benchmark for vision language models. InThe Fourteenth International Conference on Learning Representations, 2026. URLhttps://openreview.net/ forum?id=6nZKT2rL0H

2026
[21]

A. S. Kanade and T. Ganu. Do you see me : A multidimensional benchmark for evaluating visual perception in multimodal LLMs. In V. Demberg, K. Inui, and L. Marquez, editors,Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7285–7326, Rabat, Morocco, Mar. 2026. Associat...

work page doi:10.18653/v1/2026.eacl-long.343 2026
[22]

P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, H. Cheng, K.-W. Chang, M. Galley, and J. Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. InThe Twelfth International Conference on Learning Representations, 2024

2024
[23]

F. Meng, J. Wang, C. Li, Q. Lu, H. Tian, J. Liao, X. Zhu, J. Dai, Y. Qiao, P. Luo, K. Zhang, and W. Shao. Mmiu: Multimodal multi-image understanding for evaluating large vision-language models, 2024. URLhttps://arxiv.org/abs/2408.02718

work page arXiv 2024
[24]

S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? InProceedings of the 2022 conference on empirical methods in natural language processing, pages 11048–11064, 2022

2022
[25]

Qwen3.5: Towards native multimodal agents, February 2026

Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URL https: //qwen.ai/blog?id=qwen3.5

2026
[26]

Y. Ren, K. Tertikas, S. Maiti, J. Han, T. Zhang, S. Süsstrunk, and F. Kokkinos. Vgrp-bench: Visual grid reasoning puzzle benchmark for large vision-language models.arXiv preprint arXiv:2503.23064, 2025

work page arXiv 2025
[27]

Saxena, A

R. Saxena, A. P. Gema, and P. Minervini. Lost in time: Clock and calendar understanding challenges in multimodal llms, 2025. URLhttps://arxiv.org/abs/2502.05092

work page arXiv 2025
[28]

OpenAI GPT-5 System Card

A. Singh, A. Fry, A. Perelman, A. Tart, A. Ganesh, A. El-Kishky, A. McLaughlin, A. Low, A. Ostrow, A. Ananthram, et al. Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025
[29]

Tang and M

Z. Tang and M. Kejriwal. Grasp: A grid-based benchmark for evaluating commonsense spatial reasoning, 2025. URLhttps://arxiv.org/abs/2407.01892. 13 The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space

work page arXiv 2025
[30]

K. Team, T. Bai, Y. Bai, Y. Bao, S. Cai, Y. Cao, Y. Charles, H. Che, C. Chen, G. Chen, et al. Kimi k2. 5: Visual agentic intelligence.arXiv preprint arXiv:2602.02276, 2026

work page internal anchor Pith review Pith/arXiv arXiv 2026
[31]

S. Tong, E. L. B. II, P. Wu, S. Woo, A. J. IYER, S. C. Akula, S. Yang, J. Yang, M. Middepogu, Z. Wang, X. Pan, R. Fergus, Y. LeCun, and S. Xie. Cambrian-1: A fully open, vision-centric exploration of multimodal LLMs. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URLhttps://openreview.net/forum?id=Vi8AepAXGy

2024
[32]

S. Tong, Z. Liu, Y. Zhai, Y. Ma, Y. LeCun, and S. Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9568–9578, 2024

2024
[33]

F. Wang, X. Fu, J. Y. Huang, Z. Li, Q. Liu, X. Liu, M. D. Ma, N. Xu, W. Zhou, K. Zhang, et al. Muirbench: A comprehensive benchmark for robust multi-image understanding.arXiv preprint arXiv:2406.09411, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024
[34]

Z. Wang, M. Xia, L. He, H. Chen, Y. Liu, R. Zhu, K. Liang, X. Wu, H. Liu, S. Malladi, A. Chevalier, S. Arora, and D. Chen. Charxiv: Charting gaps in realistic chart understanding in multimodal llms.arXiv preprint arXiv:2406.18521, 2024

work page arXiv 2024
[35]

J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. Chain-of- thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

2022
[36]

Grok-4 model card

xAI. Grok-4 model card. 2025. URLhttps://x.ai/news/grok-4

2025
[37]

J. Xia, Y. Zang, P. Gao, S. Li, and K. Zhou. Visionary-r1: Mitigating shortcuts in visual reasoning with reinforcement learning, 2025. URLhttps://arxiv.org/abs/2505.14677

work page arXiv 2025
[38]

W. Xu, J. Wang, W. Wang, Z. Chen, W. Zhou, A. Yang, L. Lu, H. Li, X. Wang, X. Zhu, W. Wang, J. Dai, and J. Zhu. Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models.arXiv preprint arXiv:2504.15279, 2025. URLhttps://arxiv.org/abs/ 2504.15279

work page arXiv 2025
[39]

Z. Xu, C. Liu, Q. Wei, J. Wu, J. Zou, X. E. Wang, Y. Zhou, and S. Liu. More thinking, less seeing? assessing amplified hallucination in multimodal reasoning models. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

2025
[40]

J. Ye, D. Jiang, J. He, B. Zhou, Z. Huang, Z. Yan, H. Li, C. He, and W. Li. BLINK-twice: You see, but do you observe? a reasoning benchmark on visual perception. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2026. URL https://openreview.net/forum?id=g0AMmWiHCq

2026
[41]

X. Yue, Y. Ni, K. Zhang, T. Zheng, R. Liu, G. Zhang, S. Stevens, D. Jiang, W. Ren, Y. Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9556–9567, 2024

2024
[42]

X. Yue, T. Zheng, Y. Ni, Y. Wang, K. Zhang, S. Tong, Y. Sun, B. Yu, G. Zhang, H. Sun, et al. Mmmu- pro: A more robust multi-discipline multimodal understanding benchmark. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15134–15186, 2025

2025
[43]

Zetzsche, G

C. Zetzsche, G. Krieger, and B. Wegmann. The atoms of vision: Cartesian or polar?Journal of the Optical Society of America A, 16(7):1554–1565, 1999. 14 The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space

1999
[44]

Zheng, W.-L

L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in neural information processing systems, 36:46595–46623, 2023

2023
[45]

K. Zou, Z. Huang, Y. Dong, S. Tian, D. Zheng, H. Liu, J. He, B. Liu, Y. Qiao, and Z. Liu. Uni-mmmu: A massive multi-discipline multimodal unified benchmark, 2026. URLhttps: //arxiv.org/abs/2510.13759. 15 The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space A. Benchmark Details A.1. Task Taxonomy The Polaris-Bench comprises 53 pro...

work page internal anchor Pith review Pith/arXiv arXiv 2026
[46]

Visual clarity:Is the rendered image visually clear and legible? Can all relevant elements (labels, shapes, paths, grid lines) be unambiguously perceived?
[47]

Logical correctness:Is the task logic (rules, constraints, problem formulation) consistent and well-defined?
[48]

row 2”, “column 3

Answer correctness:Is the provided ground-truth answer correct? (Options:Correct,Incorrect,I don’t know.) Together with their problem solving thinking trace description. C. Evaluation Setup Details C.1. Model Query Details All models are evaluated via their publicly available APIs using default generation configurations to ensure optimal performance. We s...
[49]

Use above examples as reference

Think step-by-step about the spatial layout and rules. Use above examples as reference
[50]

Correct" or

Output your final answer clearly on the very last line. Ensure your response strictly follows this structure: <Analysis> (Your step-by-step reasoning here) </Analysis> (Your final exact answer here, on the absolute last line) [Question] {Question} <img> Figure 11|Prompt template used in the few-shot in context learning. Random Baseline.We additionally rep...

[1] [1]

M. AI. Mistral-small-3.1-24b-instruct-2503, March 2025. URLhttps://mistral.ai/news/ mistral-small-3-1

2025

[2] [2]

Anthropic

A. Anthropic. The claude 3 model family: Opus, sonnet, haiku.Claude-3 Model Card, 1(1):4, 2024

2024

[3] [3]

G. B. Arfken, H. J. Weber, and F. E. Harris.Mathematical methods for physicists: a comprehensive guide. Academic press, 2011

2011

[4] [4]

Asadi, J

M. Asadi, J. W. O’Sullivan, F. Cao, T. Nedaee, K. Rajabalifardi, F.-F. Li, E. Adeli, and E. Ashley. Mirage: The illusion of visual understanding, 2026. URLhttps://arxiv.org/abs/2603. 21687

2026

[5] [5]

M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges.arXiv preprint arXiv:2104.13478, 2021

work page internal anchor Pith review Pith/arXiv arXiv 2021

[6] [6]

J. Chen, T. Liang, S. Siu, Z. Wang, K. Wang, Y. Wang, Y. Ni, W. Zhu, Z. Jiang, B. Lyu, D. Jiang, X. He, Y. Liu, H. Hu, X. Yue, and W. Chen. Mega-bench: Scaling multimodal evaluation to over 500 real-world tasks. 2025

2025

[7] [7]

L. Chen, J. Li, X. Dong, P. Zhang, Y. Zang, Z. Chen, H. Duan, J. Wang, Y. Qiao, D. Lin, et al. Are we on the right way for evaluating large vision-language models?Advances in Neural Information Processing Systems, 37:27056–27087, 2024

2024

[8] [8]

L. Chen, W. Xie, Y. Liang, H. He, H. Zhao, Z. Yang, Z. Huang, H. Wu, H. Lu, Y. charles, Y. Bao, Y. Fan, G. Li, H. Shen, X. Chen, W. Xu, S. Si, Z. Cai, W. Chai, Z. Huang, F. Liu, T. Liu, B. Chang, X. Hu, K. Chen, Y. Ren, Y. Liu, Y. Gong, and K. Li. Babyvision: Visual reasoning beyond language,

[9] [9]

URLhttps://arxiv.org/abs/2601.06521

work page arXiv

[10] [10]

Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities

G.Comanici, E.Bieber, M.Schaekermann, I.Pasupat, N.Sachdeva, I.Dhillon, M.Blistein, O.Ram, D. Zhang, E. Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodal- ity, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025

[11] [11]

DeepMind

G. DeepMind. Gemma 4: Our most intelligent open models to date. https://blog. google/innovation-and-ai/technology/developers-tools/gemma-4/, 2026. Ac- cessed: 2026-05-01

2026

[12] [12]

Q. Dong, L. Li, D. Dai, C. Zheng, J. Ma, R. Li, H. Xia, J. Xu, Z. Wu, T. Liu, B. Chang, X. Sun, L. Li, and Z. Sui. A survey on in-context learning, 2024. URLhttps://arxiv.org/abs/2301. 00234

2024

[13] [13]

BLINK: Multimodal Large Language Models Can See but Not Perceive

X.Fu, Y.Hu, B.Li, Y.Feng, H.Wang, X.Lin, D.Roth, N.A.Smith, W.-C.Ma, andR.Krishna. Blink: Multimodal large language models can see but not perceive.arXiv preprint arXiv:2404.12390, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024

[14] [14]

Geirhos, J.-H

R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann. Shortcut learning in deep neural networks.Nature Machine Intelligence, 2(11):665–673, 2020

2020

[15] [15]

Gemini 3 flash: Frontier intelligence built for speed

Gemini Team and Google DeepMind. Gemini 3 flash: Frontier intelligence built for speed. Technical blog / model release, Google, 12 2025. URL https://blog.google/ products-and-platforms/products/gemini/gemini-3-flash/. 12 The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space

2025

[16] [16]

Gemini 3.1 flash-lite: Built for intelli- gence at scale

Gemini Team and Google DeepMind. Gemini 3.1 flash-lite: Built for intelli- gence at scale. Technical blog / model release, Google, 3 2026. URL https: //blog.google/innovation-and-ai/models-and-research/gemini-models/ gemini-3-1-flash-lite/

2026

[17] [17]

Gemini 3 pro technical report

Gemini Team, Google. Gemini 3 pro technical report. 2025. URL https://deepmind. google/models/gemini/pro/

2025

[18] [18]

J. Gu, X. Jiang, Z. Shi, H. Tan, X. Zhai, C. Xu, W. Li, Y. Shen, S. Ma, H. Liu, et al. A survey on llm-as-a-judge.The Innovation, 2024

2024

[19] [19]

Y. Hao, J. Gu, H. W. Wang, L. Li, Z. Yang, L. Wang, and Y. Cheng. Can MLLMs reason in multimodality? EMMA: An enhanced multimodal reasoning benchmark. InForty-second In- ternational Conference on Machine Learning, 2025. URLhttps://openreview.net/forum? id=v26vwjxOEz

2025

[20] [20]

M. Jia, Z. Qi, S. Zhang, W. Zhang, X. Yu, J. He, H. Wang, and L. Yi. Omnispatial: Towards comprehensive spatial reasoning benchmark for vision language models. InThe Fourteenth International Conference on Learning Representations, 2026. URLhttps://openreview.net/ forum?id=6nZKT2rL0H

2026

[21] [21]

A. S. Kanade and T. Ganu. Do you see me : A multidimensional benchmark for evaluating visual perception in multimodal LLMs. In V. Demberg, K. Inui, and L. Marquez, editors,Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7285–7326, Rabat, Morocco, Mar. 2026. Associat...

work page doi:10.18653/v1/2026.eacl-long.343 2026

[22] [22]

P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, H. Cheng, K.-W. Chang, M. Galley, and J. Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. InThe Twelfth International Conference on Learning Representations, 2024

2024

[23] [23]

F. Meng, J. Wang, C. Li, Q. Lu, H. Tian, J. Liao, X. Zhu, J. Dai, Y. Qiao, P. Luo, K. Zhang, and W. Shao. Mmiu: Multimodal multi-image understanding for evaluating large vision-language models, 2024. URLhttps://arxiv.org/abs/2408.02718

work page arXiv 2024

[24] [24]

S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? InProceedings of the 2022 conference on empirical methods in natural language processing, pages 11048–11064, 2022

2022

[25] [25]

Qwen3.5: Towards native multimodal agents, February 2026

Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URL https: //qwen.ai/blog?id=qwen3.5

2026

[26] [26]

Y. Ren, K. Tertikas, S. Maiti, J. Han, T. Zhang, S. Süsstrunk, and F. Kokkinos. Vgrp-bench: Visual grid reasoning puzzle benchmark for large vision-language models.arXiv preprint arXiv:2503.23064, 2025

work page arXiv 2025

[27] [27]

Saxena, A

R. Saxena, A. P. Gema, and P. Minervini. Lost in time: Clock and calendar understanding challenges in multimodal llms, 2025. URLhttps://arxiv.org/abs/2502.05092

work page arXiv 2025

[28] [28]

OpenAI GPT-5 System Card

A. Singh, A. Fry, A. Perelman, A. Tart, A. Ganesh, A. El-Kishky, A. McLaughlin, A. Low, A. Ostrow, A. Ananthram, et al. Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025

[29] [29]

Tang and M

Z. Tang and M. Kejriwal. Grasp: A grid-based benchmark for evaluating commonsense spatial reasoning, 2025. URLhttps://arxiv.org/abs/2407.01892. 13 The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space

work page arXiv 2025

[30] [30]

K. Team, T. Bai, Y. Bai, Y. Bao, S. Cai, Y. Cao, Y. Charles, H. Che, C. Chen, G. Chen, et al. Kimi k2. 5: Visual agentic intelligence.arXiv preprint arXiv:2602.02276, 2026

work page internal anchor Pith review Pith/arXiv arXiv 2026

[31] [31]

S. Tong, E. L. B. II, P. Wu, S. Woo, A. J. IYER, S. C. Akula, S. Yang, J. Yang, M. Middepogu, Z. Wang, X. Pan, R. Fergus, Y. LeCun, and S. Xie. Cambrian-1: A fully open, vision-centric exploration of multimodal LLMs. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URLhttps://openreview.net/forum?id=Vi8AepAXGy

2024

[32] [32]

S. Tong, Z. Liu, Y. Zhai, Y. Ma, Y. LeCun, and S. Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9568–9578, 2024

2024

[33] [33]

F. Wang, X. Fu, J. Y. Huang, Z. Li, Q. Liu, X. Liu, M. D. Ma, N. Xu, W. Zhou, K. Zhang, et al. Muirbench: A comprehensive benchmark for robust multi-image understanding.arXiv preprint arXiv:2406.09411, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024

[34] [34]

Z. Wang, M. Xia, L. He, H. Chen, Y. Liu, R. Zhu, K. Liang, X. Wu, H. Liu, S. Malladi, A. Chevalier, S. Arora, and D. Chen. Charxiv: Charting gaps in realistic chart understanding in multimodal llms.arXiv preprint arXiv:2406.18521, 2024

work page arXiv 2024

[35] [35]

J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. Chain-of- thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

2022

[36] [36]

Grok-4 model card

xAI. Grok-4 model card. 2025. URLhttps://x.ai/news/grok-4

2025

[37] [37]

J. Xia, Y. Zang, P. Gao, S. Li, and K. Zhou. Visionary-r1: Mitigating shortcuts in visual reasoning with reinforcement learning, 2025. URLhttps://arxiv.org/abs/2505.14677

work page arXiv 2025

[38] [38]

W. Xu, J. Wang, W. Wang, Z. Chen, W. Zhou, A. Yang, L. Lu, H. Li, X. Wang, X. Zhu, W. Wang, J. Dai, and J. Zhu. Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models.arXiv preprint arXiv:2504.15279, 2025. URLhttps://arxiv.org/abs/ 2504.15279

work page arXiv 2025

[39] [39]

Z. Xu, C. Liu, Q. Wei, J. Wu, J. Zou, X. E. Wang, Y. Zhou, and S. Liu. More thinking, less seeing? assessing amplified hallucination in multimodal reasoning models. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

2025

[40] [40]

J. Ye, D. Jiang, J. He, B. Zhou, Z. Huang, Z. Yan, H. Li, C. He, and W. Li. BLINK-twice: You see, but do you observe? a reasoning benchmark on visual perception. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2026. URL https://openreview.net/forum?id=g0AMmWiHCq

2026

[41] [41]

X. Yue, Y. Ni, K. Zhang, T. Zheng, R. Liu, G. Zhang, S. Stevens, D. Jiang, W. Ren, Y. Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9556–9567, 2024

2024

[42] [42]

X. Yue, T. Zheng, Y. Ni, Y. Wang, K. Zhang, S. Tong, Y. Sun, B. Yu, G. Zhang, H. Sun, et al. Mmmu- pro: A more robust multi-discipline multimodal understanding benchmark. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15134–15186, 2025

2025

[43] [43]

Zetzsche, G

C. Zetzsche, G. Krieger, and B. Wegmann. The atoms of vision: Cartesian or polar?Journal of the Optical Society of America A, 16(7):1554–1565, 1999. 14 The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space

1999

[44] [44]

Zheng, W.-L

L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in neural information processing systems, 36:46595–46623, 2023

2023

[45] [45]

K. Zou, Z. Huang, Y. Dong, S. Tian, D. Zheng, H. Liu, J. He, B. Liu, Y. Qiao, and Z. Liu. Uni-mmmu: A massive multi-discipline multimodal unified benchmark, 2026. URLhttps: //arxiv.org/abs/2510.13759. 15 The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space A. Benchmark Details A.1. Task Taxonomy The Polaris-Bench comprises 53 pro...

work page internal anchor Pith review Pith/arXiv arXiv 2026

[46] [46]

Visual clarity:Is the rendered image visually clear and legible? Can all relevant elements (labels, shapes, paths, grid lines) be unambiguously perceived?

[47] [47]

Logical correctness:Is the task logic (rules, constraints, problem formulation) consistent and well-defined?

[48] [48]

row 2”, “column 3

Answer correctness:Is the provided ground-truth answer correct? (Options:Correct,Incorrect,I don’t know.) Together with their problem solving thinking trace description. C. Evaluation Setup Details C.1. Model Query Details All models are evaluated via their publicly available APIs using default generation configurations to ensure optimal performance. We s...

[49] [49]

Use above examples as reference

Think step-by-step about the spatial layout and rules. Use above examples as reference

[50] [50]

Correct" or

Output your final answer clearly on the very last line. Ensure your response strictly follows this structure: <Analysis> (Your step-by-step reasoning here) </Analysis> (Your final exact answer here, on the absolute last line) [Question] {Question} <img> Figure 11|Prompt template used in the few-shot in context learning. Random Baseline.We additionally rep...