Pith. sign in

REVIEW 2 major objections 60 cited by

Gemma 3 adds vision and 128K context to the Gemma family while a new post-training recipe makes its 4B model competitive with the prior 27B version on math and chat tasks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · grok-4.3

2026-05-22 22:12 UTC

load-bearing objection Gemma 3 is a straightforward model release that adds vision and tweaks the attention ratio for longer context, with the usual self-reported benchmark claims that the released weights will let people check. the 2 major comments →

arxiv 2503.19786 v1 submitted 2025-03-25 cs.CL cs.AI

Gemma 3 Technical Report

Gemma Team: Aishwarya Kamath , Johan Ferret , Shreya Pathak , Nino Vieillard , Ramona Merhej , Sarah Perrin , Tatiana Matejovicova , Alexandre Ram\'e
show 202 more authors
Morgane Rivi\`ere Louis Rouillard Thomas Mesnard Geoffrey Cideron Jean-bastien Grill Sabela Ramos Edouard Yvinec Michelle Casbon Etienne Pot Ivo Penchev Ga\"el Liu Francesco Visin Kathleen Kenealy Lucas Beyer Xiaohai Zhai Anton Tsitsulin Robert Busa-Fekete Alex Feng Noveen Sachdeva Benjamin Coleman Yi Gao Basil Mustafa Iain Barr Emilio Parisotto David Tian Matan Eyal Colin Cherry Jan-Thorsten Peter Danila Sinopalnikov Surya Bhupatiraju Rishabh Agarwal Mehran Kazemi Dan Malkin Ravin Kumar David Vilar Idan Brusilovsky Jiaming Luo Andreas Steiner Abe Friesen Abhanshu Sharma Abheesht Sharma Adi Mayrav Gilady Adrian Goedeckemeyer Alaa Saade Alexander Kolesnikov Alexei Bendebury Alvin Abdagic Amit Vadi Andr\'as Gy\"orgy Andr\'e Susano Pinto Anil Das Ankur Bapna Antoine Miech Antoine Yang Antonia Paterson Ashish Shenoy Ayan Chakrabarti Bilal Piot Bo Wu Bobak Shahriari Bryce Petrini Charlie Chen Charline Le Lan Christopher A. Choquette-Choo CJ Carey Cormac Brick Daniel Deutsch Danielle Eisenbud Dee Cattle Derek Cheng Dimitris Paparas Divyashree Shivakumar Sreepathihalli Doug Reid Dustin Tran Dustin Zelle Eric Noland Erwin Huizenga Eugene Kharitonov Frederick Liu Gagik Amirkhanyan Glenn Cameron Hadi Hashemi Hanna Klimczak-Pluci\'nska Harman Singh Harsh Mehta Harshal Tushar Lehri Hussein Hazimeh Ian Ballantyne Idan Szpektor Ivan Nardini Jean Pouget-Abadie Jetha Chan Joe Stanton John Wieting Jonathan Lai Jordi Orbay Joseph Fernandez Josh Newlan Ju-yeong Ji Jyotinder Singh Kat Black Kathy Yu Kevin Hui Kiran Vodrahalli Klaus Greff Linhai Qiu Marcella Valentine Marina Coelho Marvin Ritter Matt Hoffman Matthew Watson Mayank Chaturvedi Michael Moynihan Min Ma Nabila Babar Natasha Noy Nathan Byrd Nick Roy Nikola Momchev Nilay Chauhan Oskar Bunyan Pankil Botarda Paul Caron Paul Kishan Rubenstein Phil Culliton Philipp Schmid Pier Giuseppe Sessa Pingmei Xu Piotr Stanczyk Pouya Tafti Rakesh Shivanna Renjie Wu Renke Pan Reza Rokni Rob Willoughby Rohith Vallu Ryan Mullins Sammy Jerome Sara Smoot Sertan Girgin Shariq Iqbal Shashir Reddy Shruti Sheth Siim P\~oder Sijal Bhatnagar Sindhu Raghuram Panyam Sivan Eiger Susan Zhang Tianqi Liu Trevor Yacovone Tyler Liechty Uday Kalra Utku Evci Vedant Misra Vincent Roseberry Vlad Feinberg Vlad Kolesnikov Woohyun Han Woosuk Kwon Xi Chen Yinlam Chow Yuvein Zhu Zichuan Wei Zoltan Egyed Victor Cotruta Minh Giang Phoebe Kirk Anand Rao Jessica Lo Erica Moreira Luiz Gustavo Martins Omar Sanseviero Lucas Gonzalez Zach Gleicher Tris Warkentin Vahab Mirrokni Evan Senter Eli Collins Joelle Barral Zoubin Ghahramani Raia Hadsell Yossi Matias D. Sculley Slav Petrov Noah Fiedel Noam Shazeer Oriol Vinyals Jeff Dean Demis Hassabis Koray Kavukcuoglu Clement Farabet Elena Buchatskaya Jean-Baptiste Alayrac Rohan Anil Dmitry (Dima) Lepikhin Sebastian Borgeaud Olivier Bachem Armand Joulin Alek Andreev Cassidy Hardin Robert Dadashi L\'eonard Hussenot
This is my paper
classification cs.CL cs.AI
keywords gemmamultimodallong contextpost-trainingdistillationinstruction followingmultilingualvision
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Gemma 3 models from 1B to 27B parameters that add vision understanding, support for more languages, and at least 128K token context length. It changes the architecture by increasing the ratio of local to global attention layers with short local spans to keep KV-cache memory manageable at long contexts. The models use distillation during training and apply a novel post-training recipe that boosts math, chat, instruction-following, and multilingual performance. This produces the result that the 4B instruction-tuned version matches the prior 27B model and the 27B version reaches parity with Gemini-1.5-Pro on the reported benchmarks. All models are released openly.

Core claim

Gemma 3 models achieve superior performance to Gemma 2 for both pre-trained and instruction finetuned versions. In particular, the novel post-training recipe significantly improves the math, chat, instruction-following and multilingual abilities, making Gemma3-4B-IT competitive with Gemma2-27B-IT and Gemma3-27B-IT comparable to Gemini-1.5-Pro across benchmarks.

What carries the argument

The novel post-training recipe that improves math, chat, instruction-following and multilingual abilities after distillation training.

Load-bearing premise

The chosen benchmarks and evaluation protocols accurately reflect real-world gains in math, chat, and multilingual performance without post-hoc selection or overfitting to the test sets.

What would settle it

New benchmarks or held-out tasks where Gemma3-4B-IT fails to match Gemma2-27B-IT on math and chat metrics would falsify the central performance claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The manuscript introduces Gemma 3, a family of multimodal open models (1B–27B parameters) adding vision understanding, broader language coverage, and ≥128K-token context. It modifies the architecture by raising the local-to-global attention layer ratio with short local spans to reduce KV-cache memory, trains via distillation, and applies a novel post-training recipe claimed to improve math, chat, instruction-following, and multilingual performance. This positions Gemma3-4B-IT as competitive with Gemma2-27B-IT and Gemma3-27B-IT as comparable to Gemini-1.5-Pro across benchmarks, with all models released.

Significance. If the empirical results hold under scrutiny, the work is significant for releasing capable open multimodal models that approach closed frontier performance and for practical advances in long-context efficiency via attention architecture. The model release itself and the post-training recipe constitute concrete contributions to the open AI ecosystem.

major comments (2)
  1. [Abstract] Abstract: the central claim that the novel post-training recipe 'significantly improves' math, chat, instruction-following, and multilingual abilities (making 4B competitive with 27B and 27B comparable to Gemini-1.5-Pro) is stated without any benchmark scores, error bars, dataset splits, or evaluation-protocol details. This directly underpins the headline competitiveness assertions and requires explicit quantitative support.
  2. [Results / Evaluation] Results / Evaluation sections: absence of variance estimates, exact test-set descriptions, or controls against post-hoc benchmark selection leaves the robustness of the reported gains (and the claim that they reflect genuine capability improvements rather than overfitting) unverified, which is load-bearing for the post-training contribution.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive review. The comments highlight important areas for strengthening the presentation of our results and claims. We address each major comment below and indicate the revisions we will make.

read point-by-point responses
  1. Referee: [Abstract] Abstract: the central claim that the novel post-training recipe 'significantly improves' math, chat, instruction-following, and multilingual abilities (making 4B competitive with 27B and 27B comparable to Gemini-1.5-Pro) is stated without any benchmark scores, error bars, dataset splits, or evaluation-protocol details. This directly underpins the headline competitiveness assertions and requires explicit quantitative support.

    Authors: We agree that the abstract would benefit from explicit quantitative support for the central claims. In the revised version we will insert a concise set of representative benchmark scores (e.g., average MMLU, GSM8K, HumanEval, and multilingual metrics) that directly illustrate the competitiveness statements, while keeping the abstract within length limits. This change will make the headline assertions immediately verifiable from the abstract itself. revision: yes

  2. Referee: [Results / Evaluation] Results / Evaluation sections: absence of variance estimates, exact test-set descriptions, or controls against post-hoc benchmark selection leaves the robustness of the reported gains (and the claim that they reflect genuine capability improvements rather than overfitting) unverified, which is load-bearing for the post-training contribution.

    Authors: The evaluation protocol follows the canonical test splits and official evaluation scripts of each public benchmark; we will expand the relevant section to list the exact dataset versions, prompt formats, and decoding parameters used. Variance estimates from multiple independent training runs are not feasible at this scale due to compute cost, which is standard practice in large-model technical reports. To address concerns about post-hoc selection, we will add a paragraph clarifying that the benchmark suite was fixed prior to final post-training and that the same suite is used for all model variants. These additions will improve transparency without altering the reported numbers. revision: partial

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper is a standard technical report on model release. It describes architecture modifications (local/global attention ratio), training with distillation, and a post-training recipe, then reports empirical benchmark scores for math, chat, instruction-following, and multilingual tasks. No equations, first-principles derivations, or 'predictions' are present that reduce by construction to fitted parameters or self-citations. Central claims rest on external benchmark evaluations rather than internal loops. Self-citations to prior Gemma papers are normal and non-load-bearing for the empirical results.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The report rests on empirical training runs and benchmark evaluations rather than new mathematical axioms or invented physical entities; no free parameters or invented entities are introduced beyond standard model hyperparameters.

pith-pipeline@v0.9.0 · 6656 in / 993 out tokens · 30440 ms · 2026-05-22T22:12:52.751129+00:00 · methodology

0 comments
read the original abstract

We introduce Gemma 3, a multimodal addition to the Gemma family of lightweight open models, ranging in scale from 1 to 27 billion parameters. This version introduces vision understanding abilities, a wider coverage of languages and longer context - at least 128K tokens. We also change the architecture of the model to reduce the KV-cache memory that tends to explode with long context. This is achieved by increasing the ratio of local to global attention layers, and keeping the span on local attention short. The Gemma 3 models are trained with distillation and achieve superior performance to Gemma 2 for both pre-trained and instruction finetuned versions. In particular, our novel post-training recipe significantly improves the math, chat, instruction-following and multilingual abilities, making Gemma3-4B-IT competitive with Gemma2-27B-IT and Gemma3-27B-IT comparable to Gemini-1.5-Pro across benchmarks. We release all our models to the community.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Decodable Is Not Grounded: A Vision-Ablation Arbiter for VLM Spatial Reasoning

    cs.CV 2026-06 unverdicted novelty 8.0

    A blank-image ablation test reveals that high probe accuracy on VLM spatial reasoning frequently reflects priors or inverted signs rather than image grounding, with horizontal grounded, vertical prior, and depth inverted.

  2. Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedback

    cs.LG 2026-06 unverdicted novelty 8.0

    Noisy expert imitation learning requires exponential samples for offline methods but polynomial for a variant of on-policy distillation under a noise condition.

  3. DataComp-VLM: Improved Open Datasets for Vision-Language Models

    cs.CV 2026-06 conditional novelty 8.0

    DataComp-VLM benchmark shows instruction-heavy data mixing outperforms filtering for VLM training, with DCVLM-Baseline achieving 63.6% on 33 tasks for 8B models (+5.4pp over FineVision).

  4. MedCUA-Bench: A Screenshot-Only Benchmark for Clinical Computer-Use Agents

    cs.AI 2026-06 unverdicted novelty 8.0

    MedCUA-Bench provides 18 clinical scenarios in 10 domains as a testbed for computer-use agents on medical UIs, with evaluations of 23 agents showing low success rates especially on real systems like OpenEMR.

  5. Looped Transformers with Layer Normalization Provably Learn the Power Method

    cs.LG 2026-05 unverdicted novelty 8.0

    Looped linear transformers with LN provably converge via GD to implement the power method on principal component prediction.

  6. Architecture Determines Observability of Transformers

    cs.LG 2026-04 unverdicted novelty 8.0

    Certain transformer architectures lose internal linear signals for decision quality during training, making observability an architecture-dependent property rather than a universal one.

  7. Lost in Translation: Do LVLM Judges Generalize Across Languages?

    cs.CL 2026-04 unverdicted novelty 8.0

    MM-JudgeBench shows substantial cross-lingual performance variance in 22 LVLM judges, with model size and architecture as poor predictors of multilingual robustness.

  8. SAHM: A Benchmark for Arabic Financial and Shari'ah-Compliant Reasoning

    cs.CL 2026-04 conditional novelty 8.0

    SAHM is the first Arabic financial benchmark with seven tasks including AAOIFI standards QA, fatwa reasoning, accounting exams, sentiment analysis, summarization, and event-cause reasoning, showing that Arabic fluency...

  9. ArgBench: Benchmarking LLMs on Computational Argumentation Tasks

    cs.CL 2026-04 unverdicted novelty 8.0

    ArgBench unifies 33 existing datasets into a standardized benchmark for testing LLMs across 46 argumentation tasks and analyzes the impact of prompting techniques and model factors on performance.

  10. VAREX: A Benchmark for Multi-Modal Structured Extraction from Documents

    cs.CV 2026-03 accept novelty 8.0

    VAREX benchmark shows structured output compliance limits models under 4B parameters more than extraction ability, with layout-preserving text giving the largest accuracy gains over images.

  11. Neural Signals Generate Clinical Notes in the Wild

    cs.LG 2026-01 unverdicted novelty 8.0

    CELM is the first EEG-to-language foundation model that generates clinical reports from variable-length EEG recordings using a new dataset of 9,922 reports paired with 11,000 hours of data from 9,048 patients.

  12. Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding

    cs.CV 2026-01 unverdicted novelty 8.0

    Molmo2 delivers state-of-the-art open-weight video VLMs with new grounding datasets and training methods that outperform prior open models and match or exceed some proprietary ones on pointing and tracking tasks.

  13. NEvo: Neural-Guided Evolutionary Video Synthesis for Dynamic Visual Selectivity

    cs.CV 2026-07 unverdicted novelty 7.0

    NEvo performs evolutionary search guided by a dynamic voxel-level encoding model to synthesize videos that maximize predicted activity in target brain ROIs, recovering known selectivities and revealing temporal dynami...

  14. EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards

    cs.CV 2026-06 unverdicted novelty 7.0

    EgoSafetyBench shows VLMs reliably spot hazard-containing videos but miss specific contextual hazards and are degraded by misleading in-scene text.

  15. Self-Study Reconsidered: The Hidden Fragility of Learning from Self-Generated QA

    cs.AI 2026-06 unverdicted novelty 7.0

    Self-generated QA supervision for language models is fragile due to non-uniform question selection and instruction compliance during answering, with mitigations that reduce compliance from 88% to 13%.

  16. RCT: A Robot-Collected Touch-Vision-Language Dataset for Tactile Generalization

    cs.RO 2026-06 accept novelty 7.0

    RCT dataset with sequence-preserving splits demonstrates that tactile-to-text models achieve only 25.1% Recall@1 on held-out materials, exposing generalization as the core challenge.

  17. Visual Semantic Entropy: Do Vision Language Models Recognize Visual Ambiguity?

    cs.CV 2026-06 unverdicted novelty 7.0

    VSE perturbs images only to probe visual ambiguity in VLMs, clusters outputs into semantic prototypes, and computes mass-weighted dispersion, outperforming prior entropy methods on five VQA benchmarks across five models.

  18. Large Language Model Teaches Visual Students: Cross-Modality Transfer of Fine-Grained Conceptual Knowledge

    cs.CV 2026-06 unverdicted novelty 7.0

    LaViD distills LLM conceptual knowledge to vision models via LLM-generated MCQ soft labels, outperforming vision-language distillation baselines on fine-grained benchmarks while improving robustness on spurious correl...

  19. DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cues

    cs.CV 2026-06 unverdicted novelty 7.0

    DiCoBench is a new high-resolution multi-image benchmark exposing large gaps between top MLLMs and human performance (98.3%) on differential and commonality visual cue perception.

  20. Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning

    cs.AI 2026-06 unverdicted novelty 7.0

    Cliff tokens are single-token failure triggers in LLM mathematical reasoning identified via adaptive statistical threshold; intervening at them recovers performance to 1.0 in resampling and yields up to +6.6 accuracy ...

  21. Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning

    cs.AI 2026-06 conditional novelty 7.0

    Cliff tokens are single tokens triggering LLM math reasoning failures, identified via adaptive z-test threshold on token potential; a taxonomy and Cliff-DPO optimization yield up to +6.6 accuracy gains.

  22. Training for the Model You Return: Improving Optimization for Iterate-Averaged Language Models

    cs.LG 2026-06 unverdicted novelty 7.0

    PACE is a clipped per-coordinate controller added to AdamW that improves the limiting error of the returned iterate average in both quadratic analysis and LM experiments.

  23. Training for the Model You Return: Improving Optimization for Iterate-Averaged Language Models

    cs.LG 2026-06 unverdicted novelty 7.0

    PACE is an AdamW wrapper derived from optimal control that improves the limiting error of the returned exponential-moving-average model in both theory and LM experiments.

  24. Trustworthy Image Authentication using Forensic Knowledge Graphs

    cs.CV 2026-06 unverdicted novelty 7.0

    Forensic Knowledge Graphs integrate forensic traces, causal dependencies, and scene links via a new authentication network and Iterative Context Refinement to outperform standard detectors and VLMs on detection, local...

  25. Measuring & Mitigating Over-Alignment for LLMs in Multilingual Criminal Law Courts

    cs.CL 2026-06 conditional novelty 7.0

    New benchmark quantifies language- and model-dependent over-alignment in criminal law LLM use and identifies abliteration as an effective mitigation with minimal performance cost.

  26. FlexServe: A Fast and Secure LLM Serving System for Mobile Devices with Flexible Resource Isolation

    cs.CR 2026-06 unverdicted novelty 7.0

    FlexServe decouples access and management of secure resources in TrustZone to enable efficient LLM inference on mobiles, reporting 10.05X TTFT speedup over basic strawman designs and 2.44X over optimized ones.

  27. FlexServe: A Fast and Secure LLM Serving System for Mobile Devices with Flexible Resource Isolation

    cs.CR 2026-06 unverdicted novelty 7.0

    FlexServe introduces recallable secure memory and NPU to enable cooperative secure LLM inference on mobile devices, reporting 10.05X TTFT speedup over a basic TrustZone strawman.

  28. GitReq: A Gold Standard Dataset for Software Quality Requirements

    cs.SE 2026-06 unverdicted novelty 7.0

    GitReq is a released dataset of 6,302 expert-labeled GitHub requirements across eight ISO 25010 categories with LLM classification baselines.

  29. FAPO: Fully Automated Prompt Optimization of Multi-Step LLM Pipelines

    cs.SE 2026-06 unverdicted novelty 7.0

    FAPO automates LLM pipeline optimization via iterative diagnosis and prompt-or-structure edits, beating GEPA baseline by +14.1 pp mean across 18 comparisons and +33.8 pp when structural changes occur.

  30. Algebraic Dead Directions in LayerNorm Transformers: A Forward-Pass-Only Diagnostic at LLM Scale

    cs.LG 2026-06 unverdicted novelty 7.0

    The normalized inverse-scale direction of LayerNorm's affine parameters is an exact algebraic kernel of the post-final-norm centred activation covariance for any input distribution in LayerNorm transformers.

  31. AMALIA-VL: A Native European Portuguese Open-Source Vision and Language Model

    cs.CV 2026-06 unverdicted novelty 7.0

    AMALIA-VL introduces the first open-source instruction-tuned LVLM natively optimized for European Portuguese via vision-language alignment, instruction tuning, preference optimization, and a pt-PT-centric data mix.

  32. Image Prompt Reconstruction Attacks on Distributed MLLM Inference Frameworks

    cs.CR 2026-06 unverdicted novelty 7.0

    First study of image prompt reconstruction attacks on distributed MLLM inference, proposing MPAA for pixel-level and IEDA for semantic reconstruction with 100% embedding extraction accuracy on four model families.

  33. TIGER: Inverting Transformer Gradients via Embedding-Subspace Distance Optimization

    cs.CR 2026-06 unverdicted novelty 7.0

    TIGER turns the low-rank attention gradient subspace into a differentiable objective for continuous embedding optimization, improving reconstruction quality and robustness over prior discrete token tests especially un...

  34. Steering Where to Listen: Instruction-Based Activation Steering Redirects Temporal Attention in Large Audio-Language Models

    cs.SD 2026-06 unverdicted novelty 7.0

    Instruction-based vector steering redirects temporal attention in LALMs to acoustically relevant regions, recovering queried sound event locations with 60.87-68.72% overlap accuracy without training.

  35. A History-Aware Visually Grounded Critic for Computer Use Agents

    cs.AI 2026-06 unverdicted novelty 7.0

    HiViG is a test-time critic that combines macro-action history summarization with visual grounding of execution coordinates to reduce short-sighted and visually erroneous actions in long-horizon GUI agents.

  36. Reason Twice: Segmentation via Candidate Discovery and Comparative Reasoning

    cs.CV 2026-06 unverdicted novelty 7.0

    Rea2Seg turns image segmentation into candidate mask discovery from MLLM attention followed by MLLM-based comparative scoring and selection, plus a new multi-dimensional reasoning benchmark ReasonSeg-SGDR.

  37. When Correct Decisions Hide Internal Stress: Decision-State Probing in Multimodal Language Models

    cs.CL 2026-06 unverdicted novelty 7.0

    S³E framework finds excess decision-state displacement under semantic stress in multimodal models despite consistent correct forced-choice behavior.

  38. When No Answer Is Correct: Diagnosing Absent Answer Detection for MLLMs in Video Understanding

    cs.AI 2026-06 accept novelty 7.0

    MLLMs fail to detect absent correct answers in video QA tasks across three evaluation settings, defaulting to distractors even with chain-of-thought prompting.

  39. Sci-Rho: A Multilingual Visually-Grounded Symbolic Benchmark for STEM Problems

    cs.CV 2026-06 unverdicted novelty 7.0

    Sci-Rho is a dynamic multilingual visually-grounded symbolic benchmark for STEM problems that reveals robustness gaps in current VLMs between average and worst-case performance.

  40. UrduMMLU: A Massive Multitask Benchmark for Urdu Language Understanding

    cs.CL 2026-06 unverdicted novelty 7.0

    UrduMMLU is a new native-source MCQ benchmark for Urdu that reveals top LLMs reach only ~90% accuracy with large gaps on region-specific humanities content.

  41. NAVI-Orbital: First In-Orbit Demonstration of a Zero-Shot Vision-Language Model for Autonomous Earth Observation

    cs.AI 2026-06 unverdicted novelty 7.0

    NAVI-Orbital performed the first claimed in-orbit demonstration of a zero-shot vision-language model for autonomous Earth observation using Gemma 3 on April 16, 2026.

  42. Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception

    cs.CV 2026-06 unverdicted novelty 7.0

    VLMs exhibit anchoring to discrete slant angles rather than graded responses across zero-shot, in-context, and fine-tuned settings, unlike human psychophysical patterns.

  43. Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception

    cs.CV 2026-06 unverdicted novelty 7.0

    VLMs across families and scales show anchoring to discrete slant angles in zero-shot and prompted settings rather than human-like graded texture-based slant perception.

  44. MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Models

    cs.CV 2026-06 unverdicted novelty 7.0

    Introduces MMBU benchmark for VLMs in biomedicine and demonstrates that established benchmarks mask perception deficiencies in evaluated models.

  45. Why Muon Outperforms Adam: A Curvature Perspective

    cs.LG 2026-06 conditional novelty 7.0

    Muon outperforms Adam by reducing curvature penalty via lower Normalized Directional Sharpness, as shown via Taylor approximation on LLM training and proven on stylized quadratic problems with heterogeneous curvature.

  46. Text-to-Image Models Need Less from Text Encoders Than You Think

    cs.CV 2026-06 unverdicted novelty 7.0

    A bag-of-position-tagged-words embedding guides text-to-image diffusion models as effectively as full contextual text embeddings from standard encoders.

  47. SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding

    cs.CL 2026-06 unverdicted novelty 7.0

    SEA-NLI benchmark shows low performance across 17 LLMs on Southeast Asian cultural NLI, mainly due to missing cultural knowledge, with gains from SEA-adapted models and culture-aware prompting.

  48. GeoDrive-Bench: Benchmarking Region-Specific Multimodal Reasoning in Autonomous Driving

    cs.CV 2026-06 unverdicted novelty 7.0

    GeoDrive-Bench is a new multimodal benchmark and distillation method for testing and improving VLMs on region-specific traffic-rule reasoning in autonomous driving across six countries.

  49. Not What, But How: A Framework for Auditing LLM Responses across Positioning, Generalization, Anthromorphism, and Maxims

    cs.CL 2026-06 unverdicted novelty 7.0

    Presents FRANZ framework and SQUARE corpus for multi-dimensional audit of LLM response framing on subjective cultural queries, applied to three models to reveal differences and couplings.

  50. Decentralized Instruction Tuning: Conflict-Aware Splitting and Weight Merging

    cs.LG 2026-06 unverdicted novelty 7.0

    MERIT enables decentralized instruction tuning via conflict-aware PCA splitting and parameter-space merging, raising average benchmark scores above joint training on multimodal and text mixtures.

  51. Attention-guided Fine-tuning of Multimodal Large Language Models Improves Chain-of-Thought Reasoning

    cs.CV 2026-06 unverdicted novelty 7.0

    Attentive-CoT is an attention-guided fine-tuning objective that improves chain-of-thought performance in multimodal LLMs by delaying answer commitment and increasing sustained visual-token access during rationale generation.

  52. OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification

    cs.LG 2026-05 unverdicted novelty 7.0

    OmniOPD replaces token-level logit matching in on-policy distillation with Monte Carlo chunk-level semantic verification and a peak-entropy scheduler.

  53. Subliminal Learning Is Steering Vector Distillation

    cs.AI 2026-05 unverdicted novelty 7.0

    Subliminal learning is steering vector distillation: a student fine-tuned on a steered teacher's outputs learns to imitate the steering vector.

  54. How Far Do Auto-Interpretation Labels Generalize: A Controlled Study Across Languages, Scripts, and Rewordings

    cs.CL 2026-05 unverdicted novelty 7.0

    Auto-interpretation labels for SAE features generalize poorly across languages and scripts, missing the same semantic content up to 4x more often in Serbian than English and more in Cyrillic than Latin despite determi...

  55. LLMs Need Encoders for Semantic IDs Too

    cs.IR 2026-05 unverdicted novelty 7.0

    PrefixMem encoder for Semantic IDs improves deepest-level accuracy by up to 46% relative and full-SID retrieval recall by up to 22% relative on Pinterest data across LLM families.

  56. FAM-Bench: A Multimodal Benchmark for Condition-Aware Food-as-Medicine Reasoning

    cs.AI 2026-05 unverdicted novelty 7.0

    FAM-Bench introduces 2500 nutrition-expert-verified multimodal instances across 13 conditions for dish suitability assessment and comparative ranking tasks.

  57. The Shape of Addition: Geometric Structures of Arithmetic in Large Language Models

    cs.LG 2026-05 unverdicted novelty 7.0

    LLM residual streams during addition form an Iso-Raw-Sum Trajectory anchored by digit semantics and modulated by continuous carry signals, with errors arising as geometric slippages across quantization thresholds in a...

  58. Every Act Has Its Price: Compressed Moral Composition in Frontier LLMs

    cs.CL 2026-05 unverdicted novelty 7.0

    Moral Trolley Arena shows frontier LLMs produce composite moral preferences that are compressed rather than additive functions of calibrated component act strengths across Moral Foundations Theory.

  59. CardioLens: Revealing the Clinical Reality Gap of MLLMs via Multi-Sequence Cardiac MRI Evaluations

    cs.CV 2026-05 unverdicted novelty 7.0

    CardioLens is a leakage-resistant CMR testbed of 473k slices and 13k QA pairs showing current MLLMs exhibit a large clinical reality gap with category-collapse failures on real workflows.

  60. AsymVLM: Asymmetric Token Pruning for Efficient Vision-Language Model Inference

    cs.LG 2026-05 unverdicted novelty 7.0

    AsymVLM introduces asymmetric token pruning for vision and text in VLMs to deliver up to 54% FLOPs reduction while matching or exceeding prior methods on localized visual tasks.

Reference graph

Works this paper leans on

54 extracted references · 54 canonical work pages · cited by 559 Pith papers · 29 internal anchors

  1. [1]

    GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

    J. Ainslie, J. Lee-Thorp, M. de Jong, Y. Zemlyan- skiy, F. Lebrón, and S. Sanghai. Gqa: Training generalized multi-query transformer models from multi-head checkpoints. arXiv preprint arXiv:2305.13245,

  2. [2]

    R. Anil, G. Pereyra, A. Passos, R. Ormandi, G. E. Dahl, and G. E. Hinton. Large scale distributed neural network training through online distil- lation. arXiv preprint arXiv:1804.03235,

  3. [3]

    R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lep- ikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen, et al. Palm 2 technical report.arXiv preprint arXiv:2305.10403,

  4. [4]

    A. Asai, J. Kasai, J. H. Clark, K. Lee, E. Choi, and H. Hajishirzi. Xor qa: Cross-lingual open- retrieval question answering. arXiv preprint arXiv:2010.11856,

  5. [5]

    Program Synthesis with Large Language Models

    J. Austin, A. Odena, M. I. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. J. Cai, M. Terry, Q. V. Le, and C. Sutton. Program synthesis with large language models.CoRR, abs/2108.07732,

  6. [6]

    Longformer: The Long-Document Transformer

    I. Beltagy, M. E. Peters, and A. Cohan. Long- former: The long-document transformer.arXiv preprint arXiv:2004.05150,

  7. [7]

    Y. Bisk, R. Zellers, R. L. Bras, J. Gao, and Y. Choi. PIQA: reasoning about physical commonsense in natural language. CoRR, abs/1911.11641,

  8. [8]

    Quantifying Memorization Across Neural Language Models

    N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang. Quantifying memo- rization across neural language models.arXiv preprint arXiv:2202.07646,

  9. [9]

    Chameleon: Mixed-Modal Early-Fusion Foundation Models

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818,

  10. [10]

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, 11 Gemma 3 Technical Report G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ry- der, M. Pavlov, A. Power, L. Kaiser, M. Bavar- ian, C. Winter, P. Tillet, F. P. Such, D. Cum- mings, M. Plapper...

  11. [11]

    S. Chen, S. Wong, L. Chen, and Y. Tian. Extend- ing context window of large language mod- els via positional interpolation.arXiv preprint arXiv:2306.15595,

  12. [12]

    Microsoft COCO Captions: Data Collection and Evaluation Server

    X.Chen, H.Fang, T.-Y.Lin, R.Vedantam, S.Gupta, P. Dollár, and C. L. Zitnick. Microsoft coco captions: Data collection and evaluation server. ArXiv, abs/1504.00325,

  13. [13]

    F. Chollet. On the measure of intelligence.arXiv preprint arXiv:1911.01547,

  14. [14]

    BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions

    C. Clark, K. Lee, M. Chang, T. Kwiatkowski, M. Collins, and K. Toutanova. Boolq: Explor- ing the surprising difficulty of natural yes/no questions. CoRR, abs/1905.10044,

  15. [15]

    Training Verifiers to Solve Math Word Problems

    K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schul- man. Training verifiers to solve math word problems. CoRR, abs/2110.14168,

  16. [16]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

    A. Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929,

  17. [17]

    Test of time: A benchmark for evaluating llms on temporal reasoning

    B. Fatemi, M. Kazemi, A. Tsitsulin, K. Malkan, J. Yim, J. Palowitch, S. Seo, J. Halcrow, and B. Perozzi. Test of time: A benchmark for evaluating llms on temporal reasoning.arXiv preprint arXiv:2406.09170,

  18. [18]

    X. Fu, Y. Hu, B. Li, Y. Feng, H. Wang, X. Lin, D. Roth, N. A. Smith, W.-C. Ma, and R. Krishna. Blink: Multimodal large language models can see but not perceive.ArXiv, abs/2404.12390,

  19. [19]

    arXiv preprint arXiv:2410.02089 , year=

    12 Gemma 3 Technical Report J. Gehring, K. Zheng, J. Copet, V. Mella, T. Cohen, and G. Synnaeve. Rlef: Grounding code llms in execution feedback with reinforcement learn- ing. arXiv preprint arXiv:2410.02089,

  20. [20]

    Gemma 2: Improving Open Language Models at a Practical Size

    Gemma Team. Gemma: Open models based on gemini research and technology, 2024a. Gemma Team. Gemma 2: Improving open lan- guage models at a practical size.arXiv preprint arXiv:2408.00118, 2024b. O. Goldman, U. Shaham, D. Malkin, S. Eiger, A. Hassidim, Y. Matias, J. Maynez, A. M. Gi- lady, J.Riesa, S.Rijhwani, L.Rimell, I.Szpektor, R. Tsarfaty, and M. Eyal. ...

  21. [21]

    Measuring Massive Multitask Language Understanding

    D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Mea- suring massive multitask language understand- ing. CoRR, abs/2009.03300,

  22. [22]

    understanding

    J. Hessel, A. Marasović, J. D. Hwang, L. Lee, J. Da, R. Zellers, R. Mankoff, and Y. Choi. Do an- droids laugh at electric sheep? humor" under- standing"benchmarksfromthenewyorkercap- tion contest. arXiv preprint arXiv:2209.06293,

  23. [23]

    Distilling the Knowledge in a Neural Network

    G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531,

  24. [24]

    RULER: What's the Real Context Size of Your Long-Context Language Models?

    C.-P. Hsieh, S. Sun, S. Kriman, S. Acharya, D. Rekesh, F. Jia, Y. Zhang, and B. Ginsburg. Ruler: What’s the real context size of your long-context language models?arXiv preprint arXiv:2404.06654,

  25. [25]

    arXiv preprint arXiv:2210.17546 , year=

    D. Ippolito, F. Tramèr, M. Nasr, C. Zhang, M. Jagielski, K. Lee, C. A. Choquette-Choo, and N. Carlini. Preventing verbatim memorization in language models gives a false sense of pri- vacy. arXiv preprint arXiv:2210.17546,

  26. [26]

    TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension

    M. Joshi, E. Choi, D. S. Weld, and L. Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. CoRR, abs/1705.03551,

  27. [27]

    GeomVerse: A systematic evaluation of large models for geometric reasoning.arXiv preprint arXiv:2312.12241, 2023

    M. Kazemi, H. Alvari, A. Anand, J. Wu, X. Chen, and R. Soricut. Geomverse: A systematic eval- uation of large models for geometric reasoning. arXiv preprint arXiv:2312.12241,

  28. [28]

    Kazemi, N

    M. Kazemi, N. Dikkala, A. Anand, P. Dević, I. Das- gupta, F. Liu, B. Fatemi, P. Awasthi, D. Guo, S. Gollapudi, and A. Qureshi. Remi: A dataset for reasoning with multiple images. ArXiv, abs/2406.09175, 2024a. M. Kazemi, Q. Yuan, D. Bhatia, N. Kim, X. Xu, V. Imbrasaite, and D. Ramachandran. Boardgameqa: A dataset for natural lan- guage reasoning with contr...

  29. [29]

    A Diagram Is Worth A Dozen Images

    A. Kembhavi, M. Salvato, E. Kolve, M. Seo, H. Ha- jishirzi, and A. Farhadi. A diagram is worth a dozen images. ArXiv, abs/1603.07396,

  30. [30]

    Kıcıman, R

    13 Gemma 3 Technical Report E. Kıcıman, R. Ness, A. Sharma, and C. Tan. Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint arXiv:2305.00050,

  31. [31]

    Tulu 3: Pushing Frontiers in Open Language Model Post-Training

    N. Lambert, J. Morrison, V. Pyatkin, S. Huang, H. Ivison, F. Brahman, L. J. V. Miranda, A. Liu, N. Dziri, S. Lyu, et al. T \" ulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124,

  32. [32]

    The Llama 3 Herd of Models

    LLaMa Team. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

  33. [33]

    GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models

    I. Mirzadeh, K. Alizadeh, H. Shahrokhi, O. Tuzel, S. Bengio, and M. Farajtabar. Gsm-symbolic: Understanding the limitations of mathemati- cal reasoning in large language models.arXiv preprint arXiv:2410.05229,

  34. [34]

    M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. F. Cooper, D. Ippolito, C. A. Choquette-Choo, E. Wallace, F. Tramèr, and K. Lee. Scal- able extraction of training data from (pro- duction) language models. arXiv preprint arXiv:2311.17035,

  35. [35]

    A. Ramé, J. Ferret, N. Vieillard, R. Dadashi, L. Hussenot, P.-L. Cedoz, P. G. Sessa, S. Girgin, A. Douillard, and O. Bachem. WARP: On the benefits of weight averaged rewarded policies, 2024a. A. Ramé, N. Vieillard, L. Hussenot, R. Dadashi, G. Cideron, O. Bachem, and J. Ferret. WARM: Onthebenefitsofweightaveragedrewardmod- els. In ICML, 2024b. 14 Gemma 3 T...

  36. [36]

    How to train data-efficient LLMs

    N. Sachdeva, B. Coleman, W.-C. Kang, J. Ni, L. Hong, E. H. Chi, J. Caverlee, J. McAuley, and D. Z. Cheng. How to train data-efficient llms. arXiv preprint arXiv:2402.09668,

  37. [37]

    WinoGrande: An Adversarial Winograd Schema Challenge at Scale

    K. Sakaguchi, R. L. Bras, C. Bhagavatula, and Y. Choi. WINOGRANDE: an adversarial winograd schema challenge at scale. CoRR, abs/1907.10641,

  38. [38]

    Sánchez, B

    E. Sánchez, B. Alastruey, C. Ropers, P. Stenetorp, M. Artetxe, and M. R. Costa-jussà. Linguini: A benchmark for language-agnostic linguistic reasoning. arXiv preprint arXiv:2409.12126 ,

  39. [39]

    M. Sap, H. Rashkin, D. Chen, R. L. Bras, and Y. Choi. Socialiqa: Commonsense reasoning about social interactions. CoRR, abs/1904.09728,

  40. [40]

    K. Shah, N. Dikkala, X. Wang, and R. Panigrahy. Causal language modeling can elicit search and reasoning capabilities on logic puzzles.arXiv preprint arXiv:2409.10502,

  41. [41]

    Singh, N

    H. Singh, N. Gupta, S. Bharadwaj, D. Tewari, and P. Talukdar. Indicgenbench: a multilin- gual benchmark to evaluate generation capabil- ities of llms on indic languages.arXiv preprint arXiv:2404.16816, 2024a. S. Singh, A. Romanou, C. Fourrier, D. I. Adelani, J. G. Ngui, D. Vila-Suero, P. Limkonchotiwat, K. Marchisio, W. Q. Leong, Y. Susanto, R. Ng, S. Lon...

  42. [42]

    G. Tyen, H. Mansoor, P. Chen, T. Mak, and V. Cărbune. Llms cannot find reasoning er- rors, but can correct them! arXiv preprint arXiv:2311.08516,

  43. [43]

    Jason Weston, Antoine Bordes, Sumit Chopra, Alexan- der M

    15 Gemma 3 Technical Report K. Vodrahalli, S. Ontanon, N. Tripuraneni, K. Xu, S. Jain, R. Shivanna, J. Hui, N. Dikkala, M. Kazemi, B. Fatemi, et al. Michelangelo: Long context evaluations beyond haystacks via latent structure queries. arXiv preprint arXiv:2409.12640,

  44. [44]

    LiveBench: A Challenging, Contamination-Limited LLM Benchmark

    C. White, S. Dooley, M. Roberts, A. Pal, B. Feuer, S. Jain, R. Shwartz-Ziv, N. Jain, K. Saiful- lah, S. Naidu, et al. Livebench: A challeng- ing, contamination-free llm benchmark.arXiv preprint arXiv:2406.19314,

  45. [45]

    International Conference on Learning Representations (ICLR) , year=

    M. Wortsman, P. J. Liu, L. Xiao, K. Everett, A.Alemi, B.Adlam, J.D.Co-Reyes, I.Gur, A.Ku- mar, R. Novak, et al. Small-scale proxies for large-scale transformer training instabilities. arXiv preprint arXiv:2309.14322,

  46. [46]

    Yamada, Y

    Y. Yamada, Y. Bao, A. K. Lampinen, J. Kasai, and I. Yildirim. Evaluating spatial understand- ing of large language models.arXiv preprint arXiv:2310.14540,

  47. [47]

    Zhang, L

    J. Zhang, L. Jain, Y. Guo, J. Chen, K. L. Zhou, S. Suresh, A. Wagenmaker, S. Sievert, T. Rogers, K. Jamieson, et al. Humor in ai: Massive scale crowd-sourced preferences and bench- marks for cartoon captioning.arXiv preprint arXiv:2406.10522,

  48. [48]

    16 Gemma 3 Technical Report Core contributors Aishwarya Kamath∗ Johan Ferret∗ Shreya Pathak∗ Nino Vieillard∗ Ramona Merhej∗ Sarah Perrin∗ Tatiana Matejovicova∗ Alexandre Ramé∗ Morgane Rivière∗ Louis Rouillard∗ Thomas Mesnard∗ Geoffrey Cideron∗ Jean-bastien Grill∗ Sabela Ramos∗ Edouard Yvinec∗ Michelle Casbon∗ Etienne Pot Ivo Penchev Gaël Liu Francesco Vis...

  49. [49]

    Evaluation details are described in Table

    We consider several standard bench- marks, namely MMLU (Hendrycks et al., 2020), MMLU-Pro (Wang et al., 2024), AGIEval (Zhong et al., 2023), MATH (Hendrycks et al., 2021), GSM8K (Cobbe et al., 2021), GPQA (Rein et al., 2023), MBPP (Austin et al., 2021), Hu- manEval (Chen et al., 2021). Evaluation details are described in Table

  50. [50]

    pre-trained models

    Overall we see a consis- tent improvement over STEM abilities across our Gemma 2 Gemma 3 2B 9B 27B 4B 12B 27B MMLU 52.2 71.2 75.2 59.6 74.5 78.6 MMLUpro 22.2 43.7 49.4 29.2 45.3 52.2 AGIE 31.6 53.1 55.1 42.1 57.4 66.2 MATH 16.4 36.4 42.1 24.2 43.3 50.0 GSM8K 25.0 70.2 74.6 38.4 71.0 82.6 GPQA Diamond 12.5 24.8 26.3 15.0 25.4 24.3 MBPP 31.0 51.2 60.8 46.0 ...

  51. [51]

    PaliGemma 2 was transferred at 896x896 resolution for the first four benchmarks, and at 448x448 resolution for the others

    VQAv2 84.8 85.8 85.8 84.1 84.9 85.1 Tally QA 80.6 82.4 82.1 79.0 81.3 81.7 Table 12| Performance of pre-trained checkpoints after fine-tuning on multi-modal benchmarks (without P&S). PaliGemma 2 was transferred at 896x896 resolution for the first four benchmarks, and at 448x448 resolution for the others. Comparison to PaliGemma 2.We fine-tune mul- timodal...

  52. [52]

    Evaluation details are described in Ta- ble

    et al., 2022), XQuAD (Artetxe et al., 2020), ECLeKTic (Goldman et al., 2025), IndicGen- Bench (Singh et al., 2024a), XOR QA (Asai et al., 2020). Evaluation details are described in Ta- ble

  53. [53]

    benchmarks evaluating at 32K and 128K sequence lengths. 8.1. Performance of IT models We report in Table 18, additional benchmarks on our IT models. Note that N2C refers to Natural2Code, the Gemini 1.0 internal held-out dataset, which uses author-generated sources in- stead of web-based information. BBEH refers to BIG-Bench Extra Hard (Kazemi et al., 2025...

  54. [54]

    Performance of IT models on video under- standing Additional multimodal evaluations

    8.2. Performance of IT models on video under- standing Additional multimodal evaluations. Gemma 3 IT models were evaluated on common vision benchmarks following the evaluation protocol of Gemini 1.5 (Gemini Team, 2024). The results are given in Table 16 when P&S is activated. 4B 12B 27B Perception Test MCVQA 50.6 54.9 58.1 ActivityNet-QA 46.3 50.4 52.8 Ta...