Pith. sign in

REVIEW 3 major objections 1 minor 3 cited by

Masking salient reasoning prefixes during distillation makes compact VLMs anchor their thinking on visual evidence.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 22:39 UTC pith:Z4QTKRP5

load-bearing objection The paper introduces token-wise salient reasoning-prefix masking plus self-paced scheduling to push distilled think-answer VLMs toward visual evidence, but the abstract supplies no numbers or ablation details so the size of the gain is unknown. the 3 major comments →

arxiv 2605.11651 v4 pith:Z4QTKRP5 submitted 2026-05-12 cs.CV cs.AIcs.CL

Hide to See: Reasoning-prefix Masking for Visual-anchored Thinking in VLM Distillation

Seonghoon Yu , Dongjun Nam , Byung-Kwan Lee , Jeany Son This is my paper
classification cs.CV cs.AIcs.CL
keywords VLM distillationreasoning-prefix maskingvisual-anchored thinkingmultimodal reasoningthink-answer VLMsvisual forgettingstudent-teacher distillation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper seeks to transfer think-answer reasoning from large vision-language models to smaller ones while preserving the use of visual information in intermediate steps. Long reasoning traces often cause students to forget visual inputs and rely on text alone. The proposed method masks high-influence reasoning prefixes in the student's output, compelling it to use visual tokens as compensation during training. Token-wise selection identifies which prefixes to mask based on their influence, and self-paced scheduling raises the masking level as the student improves. This results in better performance on reasoning benchmarks and greater visual reliance in the generated traces.

Core claim

Replacing the causal mask with a salient reasoning-prefix mask that blocks future tokens and salient reasoning cues encourages the student to rely on visual evidence as an alternative information source, leading to improved multimodal reasoning in the distilled model.

What carries the argument

Salient reasoning-prefix masking strategy, including token-wise masking of high-influence prefixes and self-paced budget scheduling based on teacher-student distribution discrepancy.

Load-bearing premise

Masking salient reasoning prefixes will cause the student to increase reliance on visual evidence without creating incoherent reasoning or spurious correlations.

What would settle it

If measurements of visual token attention or relevance scores in the student's reasoning steps show no increase after applying the masking compared to standard distillation, the central claim would not hold.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The distilled student model outperforms recent open-source VLMs, VLM distillation, and self-distillation methods on multimodal reasoning benchmarks.
  • Further analyses confirm enhanced visual utilization along the student thinking process.
  • The approach mitigates visual forgetting issues in long think-answer traces.
  • The self-paced masking adapts to distillation difficulty measured by distribution discrepancies.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This masking technique might generalize to distilling reasoning in other multimodal or even unimodal settings where forgetting of key inputs occurs.
  • Testing the method on tasks requiring fine-grained visual details could reveal if the visual anchoring improves specific perception capabilities.
  • Combining this masking with other regularization techniques might further stabilize the distillation process.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 1 minor

Summary. The manuscript introduces a think-answer distillation framework for VLMs called 'Hide to See' that applies token-wise salient reasoning-prefix masking (based on per-token influence) and self-paced masking budget scheduling (based on teacher-student distribution discrepancy) during distillation. The student is trained with a modified mask that blocks both future tokens and high-influence reasoning prefixes, with the goal of forcing greater reliance on visual evidence to compensate for masked textual cues. The authors claim this yields better performance than recent open-source VLMs, VLM distillation, and self-distillation baselines on multimodal reasoning benchmarks, along with analyses showing improved visual utilization in the student's reasoning trace.

Significance. If the performance gains and causal link to visual anchoring hold under rigorous controls, the method could provide a practical route to distilling long-horizon reasoning into compact VLMs while mitigating visual forgetting, which is a recognized bottleneck in current think-answer architectures.

major comments (3)
  1. [Abstract] Abstract: the central claim that the method 'outperforms recent open-source VLMs, VLM distillation, and self-distillation methods on multimodal reasoning benchmarks' and that 'further analyzes confirm enhanced visual utilization' is asserted without any quantitative results, baseline names, dataset identifiers, or ablation tables; this renders the primary empirical contribution unevaluable from the provided text.
  2. [Method] Method description (implicit in abstract): the mechanism leaves unspecified whether per-token influence scores are derived from teacher or student logits/gradients, whether the salient-prefix mask is applied only during distillation or also at inference, and whether any controlled probe (visual perturbation, attention rollout restricted to image tokens, or ablation that removes the mask component while retaining the schedule) isolates the visual-anchoring effect from generic regularization or longer effective context.
  3. [Experiments / Analyses] The skeptic concern lands: no evidence is supplied that the reported gains survive when the same self-paced schedule is applied without the salient-prefix component, leaving the load-bearing causal claim (masking increases visual dependence during the student's own autoregressive trace) untested.
minor comments (1)
  1. [Abstract] Abstract: 'think-answer' and 'Qwen3-VL-Thinking' are referenced without a brief definition or citation to the prior work that introduced the paradigm.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We address each major comment point by point below, clarifying our approach and committing to revisions where the manuscript can be strengthened.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that the method 'outperforms recent open-source VLMs, VLM distillation, and self-distillation methods on multimodal reasoning benchmarks' and that 'further analyzes confirm enhanced visual utilization' is asserted without any quantitative results, baseline names, dataset identifiers, or ablation tables; this renders the primary empirical contribution unevaluable from the provided text.

    Authors: We agree that the abstract would benefit from greater specificity to allow immediate evaluation of the claims. In the revised manuscript we will update the abstract to include concrete quantitative results (e.g., absolute gains on MMMU, MathVista and similar benchmarks), name the primary baselines (recent open-source VLMs and distillation methods), and reference the datasets and key ablations. The full experimental section already contains these details; the revision will make the abstract self-contained. revision: yes

  2. Referee: [Method] Method description (implicit in abstract): the mechanism leaves unspecified whether per-token influence scores are derived from teacher or student logits/gradients, whether the salient-prefix mask is applied only during distillation or also at inference, and whether any controlled probe (visual perturbation, attention rollout restricted to image tokens, or ablation that removes the mask component while retaining the schedule) isolates the visual-anchoring effect from generic regularization or longer effective context.

    Authors: We will expand the method section to explicitly state that per-token influence scores are computed from the teacher model's logits. The salient-prefix mask is applied exclusively during the distillation training phase and is not used at inference. We will also add descriptions of the existing controlled probes, including attention rollout restricted to image tokens and the ablation that isolates the mask component from the schedule, to better demonstrate the visual-anchoring mechanism. revision: yes

  3. Referee: [Experiments / Analyses] The skeptic concern lands: no evidence is supplied that the reported gains survive when the same self-paced schedule is applied without the salient-prefix component, leaving the load-bearing causal claim (masking increases visual dependence during the student's own autoregressive trace) untested.

    Authors: This concern is well-taken and directly targets the causal attribution of the masking strategy. We will add a new ablation experiment in the revised version that applies the identical self-paced schedule without the salient reasoning-prefix masking. This control will test whether the performance gains persist in the absence of the prefix-masking component and thereby strengthen evidence that the masking specifically promotes visual dependence in the student's autoregressive trace. Existing reasoning-trace analyses already indicate improved visual utilization; the additional ablation will provide the requested isolation. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation is self-contained

full rationale

The paper introduces a masking-based distillation method for VLMs that uses token-wise salient-prefix masking and self-paced scheduling to encourage visual anchoring during student reasoning. The abstract and description frame the approach as operating on external teacher distributions with standard benchmark evaluation and post-hoc analyses for visual utilization. No equations, fitted parameters renamed as predictions, self-citations as load-bearing premises, or ansatzes smuggled via prior work appear in the provided text. The central mechanism (masking to force reliance on visual tokens) is presented as an empirical intervention rather than a definitional identity or reduction to inputs by construction. This matches the default expectation of a non-circular paper.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities; all technical details remain unavailable.

pith-pipeline@v0.9.1-grok · 5803 in / 1092 out tokens · 23243 ms · 2026-06-30T22:39:12.848540+00:00 · methodology

0 comments
read the original abstract

Recent think-answer approaches in VLMs, such as Qwen3-VL-Thinking, boost reasoning performance by leveraging intermediate thinking steps before the final answer, but their computational cost becomes substantial, especially for larger VLMs. To distill such capabilities into compact think-answer VLMs, a primary objective is to improve the student's ability to utilize visual evidence throughout its reasoning trace, as long think-answer traces suffer from visual forgetting issues. To this end, we introduce a novel think-answer distillation framework that encourages the student to anchor its thinking on visual information by masking the student's salient reasoning prefixes. To compensate for such masked textual cues, the student is encouraged to rely more on visual evidence as an alternative source of information during distillation. Our masking strategies include: 1) token-wise salient reasoning-prefix masking, which masks high-influence reasoning prefixes selectively for each next-token prediction, and 2) self-paced masking budget scheduling, which gradually increases the masking scale according to distillation difficulty, measured by the discrepancy between teacher--student distributions. In the distillation phase, the student is guided by our salient reasoning-prefix mask, which blocks both future tokens and salient reasoning cues, in place of the standard causal mask used for auto-regressive language modeling. Experimental results show that our approach outperforms recent open-source VLMs, VLM distillation, and self-distillation methods on multimodal reasoning benchmarks, while further analyzes confirm enhanced visual utilization along the student thinking process.

Figures

Figures reproduced from arXiv: 2605.11651 by Byung-Kwan Lee, Dongjun Nam, Jeany Son, Seonghoon Yu.

Figure 1
Figure 1. Figure 1: The illustration of our reasoning-prefix masking during VLM distillation. With full [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Reliance on salient cues In particular, when distilling such long traces of think-answer VLMs, the student relies heavily on a small set of exposed textual cues, which receive disproportionately high attention values ( [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The illustration of Masking-KD. During distillation, the student is guided by our salient [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Evidence on visual-anchored thinking. (a) changes in visual attention as generation proceeds, and (b) an example of visual attention maps at the peak attention point in gray box [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison on visual attention map. We average the visual attention scores over the entire thinking trace. More visualizations are present in [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Prediction behavior of the student during distillation without and with our salient reasoning-prefix mask. Without a salient mask, the student uses a standard causal mask to predict the current token . With a salient mask, the student exploits more visual information to compensate for the masked salient reasoning prefix . More visualizations are provided in [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Evidence on textual shortcut learning in student. (a) the reverse KL divergence gradually decreases as reasoning prefixes accumulate, suggesting that the student relies on exposed reasoning cues to imitate the teacher. (b) When response prefixes are masked, the distillation loss is substantially amplified compared with masking other regions. C.2 Statistics of Masked Prefix Positions [PITH_FULL_IMAGE:figur… view at source ↗
Figure 8
Figure 8. Figure 8: Masked prefix distance In this section, we analyze the relative position of masked prefixes with respect to the current token (i.e., the distance from the current token to the masked prefix) over 19k teacher responses, as shown in [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: illustrates the think-answer response of our Masking-KD compared with the undistilled student (i.e., Qwen3-VL-2B-Thinking). Undistilled student produce perception errors, highlighted in red box , whereas ours show enhanced visual perception, highlighted in green box [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: More Comparison on Visual Attention Map. We average the visual attention scores over the entire thinking trace. C.5 More Prediction Behavior of the Student during Distillation. We illustrate the prediction behavior of the student during distillation without and with our salient reasoning-prefix mask in [PITH_FULL_IMAGE:figures/full_fig_p018_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: The instruction is used to prompt the teacher model to generate think-answer trajectories [PITH_FULL_IMAGE:figures/full_fig_p019_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: More Prediction Behavior of the Student during Distillation without and with our salient reasoning-prefix mask across four types of reasoning problems: (a) math, (b) STEM, (c) table, and (d) chart. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_12.png] view at source ↗

Review history (4 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients

    cs.CL 2026-06 unverdicted novelty 7.0

    ZPPO improves distillation to small vision-language models by using binary and negative candidate prompts plus a replay buffer for hard questions, outperforming standard distillation and GRPO on a 31-benchmark suite w...

  2. Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks

    cs.CL 2026-06 unverdicted novelty 6.0

    Evolution Fine-Tuning trains LLMs on 156K trajectories spanning 371 tasks to achieve 10.22% average improvement on 22 held-out optimization tasks and match SOTA on select circle-packing problems when combined with tes...

  3. Agent Explorative Policy Optimization for Multimodal Agentic Reasoning

    cs.CL 2026-05 unverdicted novelty 6.0

    AXPO addresses the Thinking-Acting Gap in agentic RL training by targeted resampling of tool calls in all-wrong subgroups, delivering +1.8pp gains over GRPO on nine multimodal benchmarks with an 8B model beating a 32B...

Reference graph

Works this paper leans on

40 extracted references · 13 canonical work pages · cited by 3 Pith papers · 9 internal anchors

  1. [1]

    On-policy distillation of language models: Learning from self-generated mistakes

    Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. In ICLR, 2024. 3, 14, 16

    2024

  2. [2]

    Qwen3-VL Technical Report

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025. 5, 8, 13, 15

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  3. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 5, 6

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  4. [4]

    Llava-kd: A framework of distilling multimodal large language models

    Yuxuan Cai, Jiangning Zhang, Haoyang He, Xinwei He, Ao Tong, Zhenye Gan, Chengjie Wang, Zhucun Xue, Yong Liu, and Xiang Bai. Llava-kd: A framework of distilling multimodal large language models. In ICCV, 2025. 1, 3, 6, 7, 9, 13, 14

    2025

  5. [5]

    Move-kd: Knowledge distillation for vlms with mixture of visual encoders

    Jiajun Cao, Yuan Zhang, Tao Huang, Ming Lu, Qizhe Zhang, Ruichuan An, Ningning Ma, and Shanghang Zhang. Move-kd: Knowledge distillation for vlms with mixture of visual encoders. InCVPR, 2025. 9

    2025

  6. [6]

    Beyond next-token alignment: Distilling multimodal large language models via token interactions.arXiv preprint arXiv:2602.09483, 2026

    Lin Chen, Xiaoke Zhao, Kun Ding, Weiwei Feng, Changtao Miao, Zili Wang, Wenxuan Guo, Ying Wang, Kaiyuan Zheng, Bo Zhang, et al. Beyond next-token alignment: Distilling multimodal large language models via token interactions.arXiv preprint arXiv:2602.09483, 2026. 1, 3, 6, 7, 9, 13, 14

    work page arXiv 2026

  7. [7]

    Align-kd: Distilling cross-modal alignment knowledge for mobile vision-language model.CVPR, 2025

    Qianhan Feng, Wenshuo Li, Tong Lin, and Xinghao Chen. Align-kd: Distilling cross-modal alignment knowledge for mobile vision-language model.CVPR, 2025. 1, 9

    2025

  8. [8]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025. 1, 9

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  9. [9]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. InICLR, 2021. 9

    2021

  10. [10]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. InNeurIPS Datasets and Benchmarks, 2021. 9

    2021

  11. [11]

    The curious case of neural text degeneration

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. InICLR, 2020. 4, 18

    2020

  12. [12]

    OpenAI o1 System Card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Hel- yar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024. 9

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  13. [13]

    Compodistill: Attention distillation for compositional reasoning in multimodal llms.ICLR, 2026

    Jiwan Kim, Kibum Kim, Sangwoo Seo, and Chanyoung Park. Compodistill: Attention distillation for compositional reasoning in multimodal llms.ICLR, 2026. 1, 3, 6, 7, 8, 9, 13, 14

    2026

  14. [14]

    Recursive think-answer process for llms and vlms

    Byung-Kwan Lee, Youngchae Chee, and Yong Man Ro. Recursive think-answer process for llms and vlms. CVPR Findings, 2026. 1

    2026

  15. [15]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. InICLR, 2024. 1, 5

    2024

  16. [16]

    Inter- gps: Interpretable geometry problem solving with formal language and symbolic reasoning

    Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. Inter- gps: Interpretable geometry problem solving with formal language and symbolic reasoning. InACL, 2021. 5

    2021

  17. [17]

    Learn to explain: Multimodal reasoning via thought chains for science question answering

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. 2022. 1

    2022

  18. [18]

    arXiv preprint arXiv:2405.20797 (2024)

    Shiyin Lu, Yang Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Han-Jia Ye. Ovis: Structural embedding alignment for multimodal large language model, 2024.arXiv preprint arXiv:2405.20797, 2024. 5, 6

    work page arXiv 2024

  19. [19]

    Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning, 2025

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning, 2025. 5 10

    2025

  20. [20]

    Introducing gpt-5.4

    OpenAI. Introducing gpt-5.4. https://openai.com/index/introducing-gpt-5-4/, 2026. 1

    2026

  21. [21]

    We-math: Does your large multimodal model achieve human-like mathematical reasoning? InACL, 2025

    Runqi Qiao, Qiuna Tan, Guanting Dong, MinhuiWu MinhuiWu, Chong Sun, Xiaoshuai Song, Jiapeng Wang, Zhuoma Gongque, Shanglin Lei, Yifan Zhang, et al. We-math: Does your large multimodal model achieve human-like mathematical reasoning? InACL, 2025. 5

    2025

  22. [22]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. 9

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  23. [23]

    Switch-kd: Visual-switch knowledge distillation for vision-language models.CVPR Findings, 2026

    Haoyi Sun, Xiaoxiao Wang, Ning Mao, Qian Wang, Lifu Mu, Wen Zheng, Tao Wei, and Wei Chen. Switch-kd: Visual-switch knowledge distillation for vision-language models.CVPR Findings, 2026. 1, 9

    2026

  24. [24]

    Gemini 3.1 pro: A smarter model for your most complex tasks, 2026

    The Gemini Team. Gemini 3.1 pro: A smarter model for your most complex tasks, 2026. 1

    2026

  25. [25]

    More thought, less accuracy? on the dual nature of reasoning in vision-language models

    Xinyu Tian, Shu Zou, Zhaoyuan Yang, Mengqi He, Fabian Waschkowski, Lukas Wesemann, Peter Tu, and Jing Zhang. More thought, less accuracy? on the dual nature of reasoning in vision-language models. ICLR, 2026. 2, 8

    2026

  26. [26]

    Attention is all you need.NeurIPS, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.NeurIPS, 2017. 3

    2017

  27. [27]

    Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning

    Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. InNeurIPS, 2025. 1, 5, 9, 19

    2025

  28. [28]

    Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning

    Haozhe Wang, Alex Su, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning. InNeurIPS, 2025. 2

    2025

  29. [29]

    InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency.arXiv preprint arXiv:2508.18265, 2025. 5, 6, 13

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  30. [30]

    Perception-aware policy optimization for multimodal reasoning

    Zhenhailong Wang, Xuehang Guo, Sofia Stoica, Haiyang Xu, Hongru Wang, Hyeonjeong Ha, Xiusi Chen, Yangyi Chen, Ming Yan, Fei Huang, et al. Perception-aware policy optimization for multimodal reasoning. InICLR, 2026. 1, 2

    2026

  31. [31]

    Sdrt: Enhance vision-language models by self-distillation with diverse reasoning traces.arXiv preprint arXiv:2503.01754, 2025

    Guande Wu, Huan Song, Yawei Wang, Qiaojing Yan, Yijun Tian, Lin Lee Cheong, and Panpan Xu. Sdrt: Enhance vision-language models by self-distillation with diverse reasoning traces.arXiv preprint arXiv:2503.01754, 2025. 9

    work page arXiv 2025

  32. [32]

    Logicvista: Multimodal llm logical reasoning benchmark in visual contexts

    Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logicvista: Multimodal llm logical reasoning benchmark in visual contexts. InICLR, 2025. 5

    2025

  33. [33]

    Mimo-vl technical report.ArXiv, abs/2506.03569,

    LLM-Core-Team Xiaomi. Mimo-vl technical report.arXiv preprint arXiv:2506.03569, 2025. 5, 6

    work page arXiv 2025

  34. [34]

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. InICCV, 2025. 1, 9

    2025

  35. [35]

    Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark

    Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, et al. Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark. InACL, 2025. 1, 5

    2025

  36. [36]

    Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? InECCV, 2024

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? InECCV, 2024. 1, 5

    2024

  37. [37]

    Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models

    Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self- distilled reasoner: On-policy self-distillation for large language models.arXiv preprint arXiv:2601.18734,

    work page internal anchor Pith review Pith/arXiv arXiv

  38. [38]

    R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model

    Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1-zero’s" aha moment" in visual reasoning on a 2b non-sft model.arXiv preprint arXiv:2503.05132, 2025. 1

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  39. [39]

    InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025. 5, 6 11 Hide to See: Reasoning-prefix Masking for Visual-anchored Thinking in VLM Distillation -App...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  40. [40]

    Guidelines: • The answer [NA] means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or ...