Pith. sign in

REVIEW 2 major objections 29 references

Yuvion VL-32B, trained with adversarial data synthesis and contrastive fine-tuning, surpasses comparably sized open-source and closed-source models on multimodal safety tasks while matching general capabilities.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · grok-4.3

2026-06-29 05:13 UTC pith:EZZHPBLU

load-bearing objection Yuvion VL introduces a targeted multimodal safety model with a new contrastive fine-tuning step, but the abstract gives no numbers and the new benchmark shares the training pipeline. the 2 major comments →

arxiv 2606.25034 v2 pith:EZZHPBLU submitted 2026-06-23 cs.CV cs.AI

Yuvion VL: A Multimodal Foundation Model for Adversarial Content and AI Safety

classification cs.CV cs.AI
keywords multimodal large language modelsAI safetyadversarial robustnesscontent moderationvision-language modelscontrastive fine-tuningsafety benchmarksrisk evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to establish that multimodal models purpose-built for content and AI safety, by treating the problem as inherently adversarial and designing every stage of data and training around robustness, can identify real-world risks more reliably than general-purpose models. Data comes from an automated pipeline that combines adversarial-aware synthesis with multi-stage quality control and domain annotations. Training follows a three-stage sequence of continued pretraining for cross-modal risk alignment, instruction tuning for production safety tasks, and reasoning tuning, capped by Confuse-then-Contrast Fine-Tuning that mines the model's own confusions to build multi-image contrastive groups. The 32B variant then leads on the introduced YVRE benchmarks. A sympathetic reader would care because general models routinely miss multimodal adversarial content, so a model that closes this gap without losing breadth could support safer deployment of vision-language systems.

Core claim

By treating safety as an adversarial and multimodal problem and building the entire pipeline around adversarial robustness, Yuvion VL produces models that achieve industry-leading safety performance on YVRE benchmarks. The 32B model surpasses comparably sized open-source models and the best closed-source commercial models on safety while maintaining comparable general capabilities. The key enablers are the automated adversarial-aware data synthesis pipeline with quality control, the three-stage training process, and Confuse-then-Contrast Fine-Tuning that enforces discrimination of fine-grained visual-semantic elements carrying different safety implications.

What carries the argument

Confuse-then-Contrast Fine-Tuning, a contrastive framework that mines model-specific confusions and constructs multi-image contrastive groups to enforce explicit discrimination of visually similar cases with different safety implications.

Load-bearing premise

The YVRE benchmarks and the automated adversarial data synthesis pipeline accurately capture real-world multimodal risks without overfitting to the training distribution or introducing unmeasured biases in sample quality.

What would settle it

An independent test set of real-world multimodal adversarial examples, collected outside the paper's synthesis pipeline and YVRE collection, on which Yuvion VL-32B does not outperform comparably sized models.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The three-stage training pipeline supports both production-grade safety performance and improved interpretability on complex tasks.
  • Automated adversarial-aware synthesis with multi-stage quality control can generate large-scale multimodal safety data augmented with reasoning annotations.
  • YVRE provides a focused collection of benchmarks for evaluating adversarial robustness alongside real-world capability requirements in content and AI safety.
  • Safety gains from the contrastive fine-tuning method do not require trade-offs against general multimodal capabilities.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The contrastive grouping technique could be tested on other vision-language tasks where fine visual distinctions carry high-stakes consequences, such as medical or industrial inspection.
  • Models trained this way might reduce the need for separate post-hoc safety layers when deployed in content moderation pipelines.
  • One could check whether the performance edge persists if the same contrastive method is applied to base models smaller than 32B parameters.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The paper introduces Yuvion VL, a family of multimodal LLMs purpose-built for content and AI safety. It describes an automated adversarial-aware data synthesis pipeline with multi-stage quality control, a three-stage training process (continued pretraining for cross-modal alignment, instruct post-training, and reasoning post-training), and a new Confuse-then-Contrast Fine-Tuning method that mines model-specific confusions to build multi-image contrastive groups. The authors also introduce the YVRE benchmark collection for evaluating safety, adversarial robustness, and real-world capabilities. The central claim is that Yuvion VL-32B achieves industry-leading safety performance, surpassing comparably sized open-source models and best closed-source models while maintaining comparable general capabilities.

Significance. If the superiority claims hold under distribution shifts and external validation, the work would represent a meaningful advance in multimodal AI safety by explicitly treating safety as an adversarial problem and introducing contrastive fine-tuning tailored to visual-semantic confusions. The three-stage pipeline and automated synthesis approach could influence future safety-focused model development, though the current presentation provides no quantitative evidence, ablations, or external benchmarks to assess whether gains are general or pipeline-specific.

major comments (2)
  1. [Abstract] Abstract: performance claims (industry-leading safety, surpassing open and closed models) are asserted without any metrics, baselines, error bars, ablation results, or comparison tables, providing no evidence to support the central claim.
  2. The YVRE benchmarks are introduced by the authors and constructed from the same automated adversarial-aware data synthesis pipeline used for training data (continued pretraining + Confuse-then-Contrast Fine-Tuning). No external validation set, third-party audit, or ablation on shifted test distributions is described, creating circularity risk where reported gains may reflect training-distribution fit rather than general robustness.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below and indicate where revisions will be made to strengthen the manuscript.

read point-by-point responses
  1. Referee: [Abstract] Abstract: performance claims (industry-leading safety, surpassing open and closed models) are asserted without any metrics, baselines, error bars, ablation results, or comparison tables, providing no evidence to support the central claim.

    Authors: The abstract provides a high-level summary of results, while the full quantitative evidence—including metrics, baselines, error bars, ablation studies, and comparison tables—is presented in the Experiments section and appendices. We agree the abstract should be more self-contained and will revise it to include key numerical results (e.g., YVRE safety scores and direct comparisons to open- and closed-source models) with references to the relevant tables. revision: yes

  2. Referee: The YVRE benchmarks are introduced by the authors and constructed from the same automated adversarial-aware data synthesis pipeline used for training data (continued pretraining + Confuse-then-Contrast Fine-Tuning). No external validation set, third-party audit, or ablation on shifted test distributions is described, creating circularity risk where reported gains may reflect training-distribution fit rather than general robustness.

    Authors: The synthesis pipeline generates candidate data, but YVRE uses strictly held-out splits with no training overlap, multi-stage quality filtering, and a combination of adversarial and real-world scenarios. We will add explicit clarification of the train/test separation, an ablation on distribution shifts, and a limitations discussion on external validation in the revised manuscript. The current design prioritizes adversarial coverage over external audit. revision: partial

Circularity Check

0 steps flagged

No significant circularity; claims rest on experimental results rather than definitional reduction

full rationale

The paper introduces YVRE as a new benchmark collection for evaluation and separately describes an automated adversarial-aware data synthesis pipeline for training data construction. No equations, self-citations, or explicit statements in the provided text equate the YVRE construction process to the training pipeline by definition or show that reported superiority reduces to a fitted parameter or self-referential input. The central performance claim is presented as an experimental outcome on open and internal evaluations, with no load-bearing step that collapses to its own inputs by construction. The derivation chain is therefore self-contained against external benchmarks within the limits of the quoted material.

Axiom & Free-Parameter Ledger

1 free parameters · 2 axioms · 0 invented entities

Review is abstract-only so ledger is necessarily incomplete; key unverified assumptions include effectiveness of automated adversarial synthesis and benchmark validity.

free parameters (1)
  • 32B model scale
    Selected as the size for the reported leading variant; no justification for choice provided in abstract.
axioms (2)
  • domain assumption Automated adversarial-aware data synthesis with multi-stage quality control produces large-scale high-quality multimodal samples
    Central to the data construction step described in the abstract.
  • domain assumption The three-stage training pipeline (continued pretraining, instruct post-training, reasoning post-training) improves safety without degrading general capabilities
    Assumed in the training description.

pith-pipeline@v0.9.1-grok · 6038 in / 1406 out tokens · 35446 ms · 2026-06-29T05:13:58.289127+00:00 · methodology

0 comments
read the original abstract

General-purpose models often struggle to reliably identify and understand real-world multimodal risks, largely due to the inherent multimodal adversarial nature of content and AI safety. We present Yuvion VL, a family of multimodal large language models purpose-built for content and AI safety, with both instruction-tuned and reasoning-oriented variants. Yuvion VL addresses this gap by treating safety as an inherently adversarial and multimodal problem and designing the entire pipeline around adversarial robustness. For data construction, we develop an automated pipeline integrating adversarial-aware data synthesis with multi-stage quality control, producing large-scale, high-quality multimodal samples augmented with domain knowledge and reasoning annotations. For training, we adopt a three-stage pipeline that includes continued pretraining for risk-concept cross-modal alignment, instruct post-training for production-grade safety tasks, and reasoning post-training for enhanced interpretability and performance in complex tasks. We further introduce Confuse-then-Contrast Fine-Tuning, a contrastive framework that mines model-specific confusions and constructs multi-image contrastive groups to enforce explicit discrimination of fine-grained visual-semantic elements, enabling the model to distinguish between visually similar cases with different safety implications in adversarial safety tasks. To support rigorous evaluation, we further introduce Yuvion VL RiskEval (YVRE), a collection of benchmarks covering diverse open and internal evaluations, with a focus on content and AI safety, adversarial robustness, and real-world capability requirements. Experiments show that Yuvion VL-32B achieves industry-leading safety performance, surpassing comparably sized open-source models and best closed-source commercial models, while maintaining comparable general capabilities.

Figures

Figures reproduced from arXiv: 2606.25034 by Benlei Cui, Bingyu Zhu, Bin Li, Bin Liu, Bin Tang, Chao Liu, Chengwen Yao, Chunyang Chai, Chuxi Xiao, Dongjie Zhang, Guanghui Wang, Guang Yang, Haidong Ding, Haiwen Hong, Hai Zhao, Haolei Xu, Hongxing Li, Huiming Zhang, Hui Xue, Jialun Chen, Jing Wang, Jinhao Chen, Kaiwen Lv Kacuila, Libin Dong, Longtao Huang, Meihui Lian, Meng Huang, Pengfei Sun, Ruijie Jian, Shaola Ren, Shaoxuan He, Shikai Qiu, Ting Ma, Wei Peng, Wei Wang, Wei Zhao, Wenjing Jiang, Wenxuan Liu, Xianfeng Li, Xiao Chen, Xiaoqian Xia, Xiaowen Xu, Xinyue Chen, Xipeng Cao, Xiufeng Huang, Xuan Jin, Yangfan Zhou, Yan Wang, Yiliang Zhang, Yujian Li, Yunqing Hu, Yupeng Cao, Zhaoyu Fan, Zhe Jiang, Zhenan Ye, Ziheng Wang, Ziqiang Zhu, Ziwen Xu.

Figure 1
Figure 1. Figure 1: Performance comparison between Yuvion VL and other models on open-source safety bench [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the automated Visual CoT production and quality-inspection pipeline for risk [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the Yuvion VL training pipeline. The pipeline consists of three stages: Continued [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of the C2FT framework. (a) Dynamic construction of a semantic confusion set [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Overview of the training pipeline for Yuvion VL Reasoning model. The pipeline consists [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Framework of the Yuvion VL RiskEval (YVRE). [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative case studies of Yuvion VL across eight risk scenarios. For each case, we compare [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 24 canonical work pages · 18 internal anchors

  1. [1]

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

    Jinze Bai, Shuai Bai, Shusheng Yang, et al. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966,

  2. [2]

    Qwen3-VL Technical Report

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, et al. Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025a. Shuai Bai, Keqin Chen, Xuejing Liu, et al. Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923, 2025b. Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, ...

  3. [3]

    InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks.arXiv preprint arXiv:2312.14238,

  4. [4]

    Llama guard 3 vision: Safeguarding human-ai image understanding conversations.arXiv preprint arXiv:2411.10414,

    Jianfeng Chi, Ujjwal Karn, Hongyuan Zhan, Eric Smith, Javier Rando, Yiming Zhang, Kate Plawiak, Zacharie Delpierre Coudert, Kartikeya Upasani, and Mahesh Pasupuleti. Llama guard 3 vision: Safeguarding human-ai image understanding conversations.arXiv preprint arXiv:2411.10414,

  5. [5]

    Boolq: Exploring the surprising difficulty of natural yes/no questions

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pp. 2924–2936,

  6. [6]

    Training Verifiers to Solve Math Word Problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,

  7. [7]

    Tc-pad\’e: Trajectory-consistent pad\’e approximation for diffusion acceleration.arXiv preprint arXiv:2603.02943, 2026a

    Benlei Cui, Shaoxuan He, Bukun Huang, Zhizeng Ye, Yunyun Sun, Longtao Huang, Hui Xue, Yang Yang, Jingqun Tang, Zhou Zhao, et al. Tc-pad\’e: Trajectory-consistent pad\’e approximation for diffusion acceleration.arXiv preprint arXiv:2603.02943, 2026a. 1Correspondence to:honghaiwen.hhw@alibaba-inc.com. 21 Benlei Cui, Bukun Huang, Zhizeng Ye, Xuemei Dong, Tuo...

  8. [8]

    AEGIS: Online adaptive ai content safety moderation with ensemble of llm experts.arXiv preprint arXiv:2404.05993,

    Shaona Ghosh, Prasoon Varshney, Erick Galinkin, and Christopher Parisien. AEGIS: Online adaptive ai content safety moderation with ensemble of llm experts.arXiv preprint arXiv:2404.05993,

  9. [9]

    WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs

    Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms.arXiv preprint arXiv:2406.18495,

  10. [10]

    Llava- Guard: An open vlm-based framework for safeguarding vision datasets and models.arXiv preprint arXiv:2406.05113,

    Lukas Helff, Felix Friedrich, Manuel Brack, Kristian Kersting, and Patrick Schramowski. Llava- Guard: An open vlm-based framework for safeguarding vision datasets and models.arXiv preprint arXiv:2406.05113,

  11. [11]

    Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, et al. Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749,

  12. [12]

    Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations

    Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. Llama guard: Llm-based input-output safeguard for human-ai conversations.arXiv preprint arXiv:2312.06674,

  13. [13]

    LLaVA-OneVision: Easy Visual Task Transfer

    Bo Li, Yuanhan Zhang, Dong Guo, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326,

  14. [14]

    SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension

    22 Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Benchmarking multimodal llms with generative comprehension.arXiv preprint arXiv:2307.16125,

  15. [15]

    YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Models

    Junyu Lin, Meizhen Liu, Xiufeng Huang, Jinfeng Li, Haiwen Hong, Xiaohan Yuan, Yuefeng Chen, Longtao Huang, Hui Xue, Ranjie Duan, et al. Yufeng-xguard: A reasoning-centric, interpretable, and flexible guardrail model for large language models.arXiv preprint arXiv:2601.15588,

  16. [16]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. InInternational Conference on Learning Representations, volume 2024, pp. 23439–23554,

  17. [17]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Machel Reid, Nikolay Savinov, Denis Teplyashin, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530,

  18. [18]

    Gemini: A Family of Highly Capable Multimodal Models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, et al. Gemini: A family of highly capable multimodal models.arXiv preprint arXiv:2312.11805,

  19. [19]

    Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

    Peng Wang, Shuai Bai, Sinan Tan, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191,

  20. [20]

    Llava-critic-r1: Your critic model is secretly a strong policy model.arXiv preprint arXiv:2509.00676, 2025a

    Xiyao Wang et al. Llava-critic-r1: Your critic model is secretly a strong policy model.arXiv preprint arXiv:2509.00676, 2025a. 23 Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, and Hao Fei. Multimodal chain-of-thought reasoning: A comprehensive survey.arXiv preprint arXiv:2503.12605, 2025b. Haoning Wu, Zicheng Zhang, Erl...

  21. [21]

    EVADE-Bench: Multimodal Benchmark for Evaluating and Enhancing Evasive Content Detection

    Ancheng Xu, Zhihao Yang, Jingpeng Li, Guanghu Yuan, Longze Chen, Liang Yan, et al. EVADE- Bench: Multimodal benchmark for evaluating and enhancing evasive content detection.arXiv preprint arXiv:2505.17654,

  22. [22]

    LLaVA-CoT: Let Vision Language Models Reason Step-by-Step

    Guowei Xu, Peng Jin, Ziang Wu, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step.arXiv preprint arXiv:2411.10440,

  23. [23]

    Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts

    Haolei Xu, Haiwen Hong, Hongxing Li, Rui Zhou, Yang Zhang, Longtao Huang, Hui Xue, Yongliang Shen, Weiming Lu, and Yueting Zhuang. Seeing but not thinking: Routing distraction in multimodal mixture-of-experts.arXiv preprint arXiv:2604.08541,

  24. [24]

    ProGuard: Towards proactive multimodal safeguard.arXiv preprint arXiv:2512.23573,

    Shaohan Yu, Lijun Li, Chenyang Si, Lu Sheng, and Jing Shao. ProGuard: Towards proactive multimodal safeguard.arXiv preprint arXiv:2512.23573,

  25. [25]

    ShieldGemma: Generative AI Content Moderation Based on Gemma

    Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, Olivia Sturman, and Oscar Wahltinez. Shieldgemma: Generative ai content moderation based on gemma.arXiv preprint arXiv:2407.21772,

  26. [26]

    R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025a. Zicheng Zhang, Haoning Wu, Chunyi Li, Yingjie Zhou, Wei Sun, Xiongkuo Min, Zijian Chen, Xiaohong Liu, Weisi ...

  27. [27]

    for diagram understanding and struc- tured visual reasoning over educational figures. • MM General VQA:MMBench(Liu et al., 2024c),MME(Fu et al., 2026),MMStar(Chen et al., 2024),SEEDBench(Li et al., 2023),ScienceQA(Lu et al., 2022),A-Bench(Zhang et al., 2025b), and Q-Bench(Wu et al.,

  28. [28]

    • Text Chinese language understanding.:C3(Sun et al., 2020),CLUEWSC(Xu et al., 2020), andXiezhi- CNfor Chinese knowledge understanding, and commonsense reasoning

    for object counting and spatial grounding. • Text Chinese language understanding.:C3(Sun et al., 2020),CLUEWSC(Xu et al., 2020), andXiezhi- CNfor Chinese knowledge understanding, and commonsense reasoning. • Text Commonsense and reading comprehension. BoolQ(Clark et al., 2019),WinoGrande(Sakaguchi et al.,

  29. [29]

    for commonsense reasoning, reading comprehension, multi-step inference, and robustness on open-ended or ambiguous problems. • Text Mathematical reasoning.This group includesGSM8K-ZH(Cobbe et al., 2021),APE210K,TAL- SCQ5K-CNfor arithmetic problem solving, formal mathematical reasoning, and theorem-related question answering in both Chinese and English. • T...