pith. sign in

arxiv: 2302.13971 · v1 · pith:ASFYTQH3new · submitted 2023-02-27 · 💻 cs.CL

LLaMA: Open and Efficient Foundation Language Models

Hugo Touvron , Thibaut Lavril , Gautier Izacard , Xavier Martinet , Marie-Anne Lachaux , Timoth\'ee Lacroix , Baptiste Rozi\`ere , Naman Goyal
show 6 more authors
Eric Hambro Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample
This is my paper

Pith reviewed 2026-05-24 09:34 UTC · model grok-4.3

classification 💻 cs.CL
keywords foundation language modelspublic datasetsmodel scalingbenchmark performanceopen releaseGPT-3 comparison
0
0 comments X

The pith

LLaMA models trained only on public data outperform GPT-3 despite fewer parameters.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces LLaMA, a family of foundation language models sized from 7B to 65B parameters. These models are trained on trillions of tokens drawn exclusively from publicly available datasets. The central result is that the 13B version surpasses GPT-3 (175B) on most standard benchmarks while the 65B version reaches parity with Chinchilla-70B and PaLM-540B. The authors release the full set of models to support wider research. This shows that high performance is achievable without access to proprietary data collections.

Core claim

It is possible to train state-of-the-art foundation language models using only publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B.

What carries the argument

The LLaMA collection of foundation language models trained on public data mixtures to reach competitive benchmark scores.

Load-bearing premise

The evaluation benchmarks used are fair and representative measures of capability that do not favor models trained on the specific public data mixtures chosen by the authors.

What would settle it

A new held-out benchmark or evaluation set on which LLaMA-13B scores below GPT-3 or LLaMA-65B falls behind Chinchilla-70B.

read the original abstract

We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces the LLaMA collection of foundation language models with 7B to 65B parameters, trained on trillions of tokens from publicly available datasets only. It reports that the 13B model outperforms GPT-3 (175B) on most benchmarks and the 65B model is competitive with Chinchilla-70B and PaLM-540B, and releases the models openly.

Significance. If the benchmark results hold after accounting for potential data issues, the work is significant in demonstrating that competitive large language models can be developed using exclusively public data, which has implications for accessibility and reproducibility in the field. The open release of models is a notable strength that enables community verification and extension.

major comments (2)
  1. [§2.1] §2.1 Pre-training data: The description of the data mixture (CommonCrawl, C4, GitHub, books, Wikipedia) does not report quantitative decontamination statistics or n-gram/document overlap measures with the test splits of the benchmarks used in §3; this directly bears on whether the headline claim that LLaMA-13B outperforms GPT-3 (175B) reflects genuine capability gains rather than memorization.
  2. [§3] §3 Main results, Tables 2–4: The reported benchmark scores lack accompanying error bars, details on the exact evaluation protocol (e.g., number of shots, prompt templates, decontamination steps per task), or multiple-run statistics, which undermines the robustness of the cross-model comparisons that form the central empirical claim.
minor comments (2)
  1. [Figure 1] Figure 1 and associated text: axis labels and legend entries for model-size scaling could be made more consistent across subplots for clarity.
  2. [Abstract] The abstract states performance claims without previewing the exact token count or data breakdown; while the body supplies these, a brief quantitative summary in the abstract would improve readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback. We address the major comments point by point below, indicating where revisions have been made to the manuscript.

read point-by-point responses

  1. Referee: [§2.1] The description of the data mixture does not report quantitative decontamination statistics or n-gram/document overlap measures with the test splits of the benchmarks used in §3; this directly bears on whether the headline claim that LLaMA-13B outperforms GPT-3 (175B) reflects genuine capability gains rather than memorization.

    Authors: We agree that quantitative decontamination statistics would provide additional reassurance. The manuscript describes the removal of overlapping documents but does not include specific overlap measures. We have revised the text in §2.1 to provide more information on the decontamination process employed during data preparation. However, detailed n-gram overlap percentages were not computed or recorded at the time, so we cannot provide them. We believe the description of the method used sufficiently addresses the spirit of the comment. revision: partial

  2. Referee: [§3] The reported benchmark scores lack accompanying error bars, details on the exact evaluation protocol (e.g., number of shots, prompt templates, decontamination steps per task), or multiple-run statistics, which undermines the robustness of the cross-model comparisons that form the central empirical claim.

    Authors: We acknowledge the value of including more evaluation details. We have updated §3 with additional information on the evaluation protocol, specifying the number of shots and referencing the prompt templates from the original benchmark papers for each task. Decontamination is now explicitly linked to the procedure in §2.1. Error bars and multiple-run statistics are not provided because the models were evaluated in a single run, which is standard practice given the computational expense of large-scale training and inference. We have added a note in the revised manuscript acknowledging this limitation. revision: partial

Circularity Check

0 steps flagged

No circularity: purely empirical benchmark reporting

full rationale

The paper presents an empirical study of training LLaMA models on public datasets and evaluating them on standard benchmarks. No derivation chain, equations, or first-principles predictions exist that could reduce to fitted inputs, self-definitions, or self-citation load-bearing steps. The central claims (e.g., LLaMA-13B outperforming GPT-3 on benchmarks) are direct performance measurements, not constructed outputs from the paper's own methods. This is a standard non-circular empirical report.

Axiom & Free-Parameter Ledger

2 free parameters · 2 axioms · 0 invented entities

The work rests on the standard transformer architecture and scaling behavior established in prior literature; no new mathematical axioms or invented entities are introduced. Training involves many unstated hyperparameters typical of large-scale language model training.

free parameters (2)
  • model parameter counts
    7B, 13B, 33B, 65B sizes chosen to explore scaling regime.
  • training token count
    Trillions of tokens selected to reach competitive performance.
axioms (2)
  • standard math Transformer architecture is sufficient for language modeling at these scales
    Invoked implicitly by training standard decoder-only transformers.
  • domain assumption Publicly available datasets contain sufficient high-quality text for SOTA performance
    Central to the claim that proprietary data is unnecessary.

pith-pipeline@v0.9.0 · 5675 in / 1352 out tokens · 26828 ms · 2026-05-24T09:34:01.387284+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Tight Sample Complexity of Transformers

    cs.LG 2026-06 unverdicted novelty 8.0

    Depth-L transformers with W parameters have VC dimension Theta(L W log(T W)), yielding matching O(L W log((T+T')W)) upper and Omega(L W log((T+T')W/L)) lower bounds on sample complexity for chain-of-thought learning.

  2. SVHalluc: Benchmarking Speech-Vision Hallucination in Audio-Visual Large Language Models

    eess.AS 2026-05 unverdicted novelty 8.0

    SVHalluc benchmark shows open-source audio-visual LLMs achieve near-random accuracy on semantic and temporal speech-vision alignment tasks while Gemini 2.5 Pro performs substantially better.

  3. Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination

    cs.CV 2026-05 unverdicted novelty 8.0

    VLMs fail to detect semantically different image swaps up to 60% of the time despite self-reflective statements, with thinking models more vulnerable and attention analysis showing self-reflection does not increase vi...

  4. Privacy Auditing with Zero (0) Training Run

    cs.CR 2026-05 unverdicted novelty 8.0

    Zero-Run auditing supplies valid lower bounds on differential privacy parameters from fixed member and non-member datasets by modeling and correcting distribution-shift confounding via causal-inference techniques.

  5. Effective Context in Transformers: An Analysis of Fragmentation and Tokenization

    cs.LG 2026-05 unverdicted novelty 8.0

    Fragmentation strictly raises optimal finite-context log-loss on Markov sources while tokenization can make a short token window equivalent to a longer source window under reliability and compression conditions.

  6. Grid Games: The Power of Multiple Grids for Quantizing Large Language Models

    cs.LG 2026-05 accept novelty 8.0

    Allowing each quantization group to select among multiple 4-bit grids improves accuracy over single-grid FP4 for both post-training and pre-training of LLMs.

  7. Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Models

    cs.LG 2026-05 unverdicted novelty 8.0

    Adaptive scheduling of interventions in discrete diffusion language models, timed to attribute-specific commitment schedules discovered with sparse autoencoders, delivers precise multi-attribute steering up to 93% str...

  8. When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds

    cs.LG 2026-05 unverdicted novelty 8.0

    SignSGD provably beats SGD by a factor of d under sparse noise via matched ℓ1-norm upper and lower bounds, with an equivalent result for Muon on matrices, and this predicts faster GPT-2 pretraining.

  9. Backdoor Attacks on Decentralised Post-Training

    cs.CR 2026-03 conditional novelty 8.0

    An adversary controlling an intermediate pipeline stage in decentralized LLM post-training can inject a backdoor that reduces alignment from 80% to 6%, with the backdoor persisting in 60% of cases even after subsequen...

  10. Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Servers

    cs.SE 2025-06 conditional novelty 8.0

    First study of 1,899 MCP servers finds eight distinct vulnerabilities (only three traditional), 7.2% with general issues, 5.5% with tool poisoning, and 66% with code smells, urging MCP-specific security practices.

  11. BEAVER: An Enterprise Benchmark for Text-to-SQL

    cs.CL 2024-09 unverdicted novelty 8.0

    BEAVER is the first text-to-SQL benchmark from private enterprise data warehouses, revealing SOTA agentic frameworks achieve only 10.8% accuracy on complex real-world queries.

  12. MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?

    cs.CV 2024-08 conditional novelty 8.0

    MME-RealWorld is the largest manually annotated high-resolution benchmark for MLLMs, where even the best models achieve less than 60% accuracy on challenging real-world tasks.

  13. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents

    cs.CR 2024-06 unverdicted novelty 8.0

    AgentDojo introduces an extensible evaluation framework populated with realistic agent tasks and security test cases to measure prompt injection robustness in tool-using LLM agents.

  14. AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments

    cs.HC 2024-05 conditional novelty 8.0

    AgentClinic is a multimodal agent benchmark demonstrating that LLM diagnostic accuracy on MedQA drops to below one-tenth in sequential clinical simulations, with Claude-3.5 leading and large tool-use differences acros...

  15. ORPO: Monolithic Preference Optimization without Reference Model

    cs.CL 2024-03 conditional novelty 8.0

    ORPO performs preference alignment during supervised fine-tuning via a monolithic odds ratio penalty, allowing 7B models to outperform larger state-of-the-art models on alignment benchmarks.

  16. Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders

    cs.IR 2024-03 unverdicted novelty 8.0

    BLaIR is a new benchmark and 570M-review dataset showing that LLM performance rankings on recommendation tasks have little correlation with rankings on general embedding benchmarks like MTEB.

  17. Mamba: Linear-Time Sequence Modeling with Selective State Spaces

    cs.LG 2023-12 unverdicted novelty 8.0

    Mamba is a linear-time sequence model using input-dependent selective SSMs that achieves SOTA results across modalities and matches twice-larger Transformers on language modeling with 5x higher inference throughput.

  18. MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI

    cs.CL 2023-11 unverdicted novelty 8.0

    MMMU provides 11.5K heterogeneous college-level multimodal questions that current models solve at 56-59% accuracy, establishing a new standard for expert multimodal evaluation.

  19. Tree of Thoughts: Deliberate Problem Solving with Large Language Models

    cs.CL 2023-05 accept novelty 8.0

    Tree of Thoughts enables language models to solve complex planning tasks by generating, evaluating, and searching over coherent intermediate thoughts in a tree, raising Game of 24 success from 4% to 74% with GPT-4.

  20. API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs

    cs.CL 2023-04 conditional novelty 8.0

    API-Bank is a new benchmark and training dataset for tool-augmented LLMs that shows fine-tuned models can approach GPT-3.5 tool-use effectiveness.

  21. Instruction Tuning with GPT-4

    cs.CL 2023-04 unverdicted novelty 8.0

    GPT-4-generated instruction data produces superior zero-shot performance in finetuned LLaMA models versus prior state-of-the-art data.

  22. HERMES: A Multi-Granularity Labeling Substrate for Pre-training Data Mixtures

    cs.LG 2026-07 unverdicted novelty 7.0

    HERMES provides a reusable hierarchical labeling substrate for pre-training data that reveals granularity-specific effects in data mixing rules during model training.

  23. OntoLearner: A Modular Python Library for Ontology Learning with Large Language Models

    cs.AI 2026-07 unverdicted novelty 7.0

    OntoLearner supplies the first cross-domain ontology collection and benchmarking infrastructure for LLM-driven ontology learning, finding that failure scales with ontological complexity instead of model size.

  24. Agentic generation of verifiable rules for deterministic, self-expanding reaction classification

    cs.AI 2026-07 unverdicted novelty 7.0

    Multi-agent LLMs generate and verify 14,073 deterministic reaction rules from 665,901 patents, enabling 97.7% classification of unseen reactions with finer resolution than fixed proprietary systems.

  25. Language-Assisted Super-Resolution from Real-World Low-Resolution Patches

    cs.CV 2026-06 unverdicted novelty 7.0

    LA-SR redefines unpaired super-resolution in language space by projecting images into a semantically rich representation and applying vision-language model guided losses to handle real-world degradations extracted fro...

  26. Probing Memorization of Tabular In-Context Learning

    cs.LG 2026-06 unverdicted novelty 7.0

    A new probing framework detects moderate parametric memorization signals in tabular in-context learning models under single-task fine-tuning, strongest on low-cardinality tasks, but signals largely disappear under rea...

  27. Search for Truth from Reasoning: A Dynamic Representation Editing Framework for Steering LLM Trajectories

    cs.AI 2026-06 unverdicted novelty 7.0

    DynaSteer dynamically steers LLM reasoning trajectories toward truth via pattern clustering, Fisher-LDA projection, and entropy-triggered representation edits, improving performance on MATH and generalizing to coding.

  28. A Sensitivity-Aware Test Collection for Search Among Personal Information

    cs.IR 2026-06 accept novelty 7.0

    A new sensitivity-labeled test collection is released from Enron emails with crowdsourced queries, relevance judgments, and LLM extensions for evaluating sensitivity-aware search.

  29. Large Language Model Teaches Visual Students: Cross-Modality Transfer of Fine-Grained Conceptual Knowledge

    cs.CV 2026-06 unverdicted novelty 7.0

    LaViD distills LLM conceptual knowledge to vision models via LLM-generated MCQ soft labels, outperforming vision-language distillation baselines on fine-grained benchmarks while improving robustness on spurious correl...

  30. An Empirical Study of LLM-Generated Specifications for VeriFast

    cs.SE 2026-06 unverdicted novelty 7.0

    LLMs preserve functional behavior in over 91% of generated VeriFast specifications and source code but achieve only 31.4% verification success, with 94% of failures due to separation logic domain knowledge errors.

  31. $\text{DT}^2$: Decision-Targeted Digital Twins

    cs.LG 2026-06 unverdicted novelty 7.0

    DT² trains digital twins to preserve pairwise policy rankings from fitted Q-evaluation on offline data rather than minimizing one-step transition errors, improving policy ranking and reducing decision regret.

  32. SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessity

    cs.CV 2026-06 unverdicted novelty 7.0

    SSMNBench shows that MLLMs suffer distraction degradation on single-view-sufficient tasks and fail to integrate geometric evidence across views, instead relying on semantic averaging and view preference.

  33. The Geometry of Sequential Learning: Lie-Bracket Prediction of Transfer Order

    cs.LG 2026-06 unverdicted novelty 7.0

    Lie brackets of gradient fields provide a geometric score for optimal transfer order in sequential learning, enabling efficient ranking of many sources via tournament methods.

  34. PatternGSL: A Structured Specification Language for Template-Free and Simulation-Ready 3D Garments

    cs.CV 2026-06 unverdicted novelty 7.0

    PatternGSL is a new template-free specification language for complete sewing patterns that enables direct single-image prediction of simulation-ready garments via a vision-language model, supported by a new 300K paire...

  35. PatternGSL: A Structured Specification Language for Template-Free and Simulation-Ready 3D Garments

    cs.CV 2026-06 unverdicted novelty 7.0

    PatternGSL introduces a learnable specification language for sewing patterns that lets vision-language models reconstruct explicit, simulation-ready 3D garments from single images, backed by a new 300K paired dataset.

  36. PatternGSL: A Structured Specification Language for Template-Free and Simulation-Ready 3D Garments

    cs.CV 2026-06 unverdicted novelty 7.0

    PatternGSL is a learnable template-free language for garment sewing patterns enabling direct VLM prediction of simulation-ready 3D garments from images, backed by a 300K image-to-specification dataset.

  37. PatternGSL: A Structured Specification Language for Template-Free and Simulation-Ready 3D Garments

    cs.CV 2026-06 unverdicted novelty 7.0

    PatternGSL defines a compact, learnable specification language for sewing patterns that enables direct image-to-structured-garment prediction via VLM without templates or post-optimization, supported by a 300K dataset.

  38. MeshFlow: Mesh Generation with Equivariant Flow Matching

    cs.GR 2026-06 unverdicted novelty 7.0

    MeshFlow applies equivariant optimal-transport flow matching to generate triangle meshes as soups, matching autoregressive quality with an 18x inference speedup.

  39. Convergence of Gradient Descent for General Neural Network Architectures Beyond the NTK Regime

    cs.LG 2026-06 unverdicted novelty 7.0

    Proves GD convergence to stationary point neighborhoods for general NN architectures beyond NTK via block-level analysis, analyticity, and local smoothness conditions.

  40. Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models

    cs.CL 2026-06 unverdicted novelty 7.0

    Entity-level membership inference determines whether information about a target real-world entity was used in LLM training, using only black-box generated text and achieving AUC up to 0.97 on person entities.

  41. Training-free Task Classification for Multi-Task Model Merging

    cs.LG 2026-06 conditional novelty 7.0

    SiM enables training-free routing in multi-task model merging by scoring test inputs via projection residuals onto SVD-based task manifolds precomputed from small support sets.

  42. UniRank: Unified Rank Allocation for Low-Rank LLM Compression

    cs.LG 2026-06 unverdicted novelty 7.0

    UniRank introduces dual local-global scoring for rank allocation in LLM low-rank decomposition plus rank-preserving fine-tuning, achieving up to 50% lower perplexity than uniform baselines in one-shot compression.

  43. GitReq: A Gold Standard Dataset for Software Quality Requirements

    cs.SE 2026-06 unverdicted novelty 7.0

    GitReq is a released dataset of 6,302 expert-labeled GitHub requirements across eight ISO 25010 categories with LLM classification baselines.

  44. Towards Understanding the Power and Limits of the Muon Optimizer: A River-Valley Perspective

    cs.LG 2026-06 unverdicted novelty 7.0

    Muon moves faster along signal river directions early but converges slower or oscillates near optima than GD due to orthogonal updates removing scale information, supporting two-stage optimization.

  45. Triage Score: A Counterfactual Risk Assessment Instrument

    stat.AP 2026-06 unverdicted novelty 7.0

    Triage scores extend risk scores via additive counterfactual utilities to incorporate intervention effects in high-stakes decisions.

  46. Shifting-based Optimizable Linear Relaxations for General Activation Functions

    cs.LG 2026-06 unverdicted novelty 7.0

    SLiR parameterizes linear relaxations by slope and uses shifting to compute sound bounds for general activation functions, enabling up to 7.8x more verified properties than prior methods.

  47. ITNet: A Learnable Integral Transform That Subsumes Convolution, Attention, and Recurrence

    cs.AI 2026-06 unverdicted novelty 7.0

    ITNet unifies convolution, self-attention, and autoregressive recurrence as special cases of a single learnable integral transform implemented via an MLP kernel and matches specialized models on ImageNet, GLUE, ModelN...

  48. RTSGameBench: An RTS Benchmark for Strategic Reasoning by Vision-Language Models

    cs.AI 2026-06 unverdicted novelty 7.0

    RTSGameBench is a new extensible benchmark for VLMs using diverse RTS matchups, diagnostic mini-games targeting individual competencies, and a self-evolving query-to-game generator, with results showing poor VLM perfo...

  49. Moving Beyond Diversity: Visual Token Pruning as Subspace Reconstruction for Efficient VLMs

    cs.CV 2026-06 unverdicted novelty 7.0

    SPARE reformulates visual token pruning as column subset selection to minimize reconstruction error and uses anti-relevance for context-aware selection in VLMs.

  50. SAGE: Retain-Aware Post-Hoc Sanitization of Final Unlearning Vector

    cs.LG 2026-06 unverdicted novelty 7.0

    SAGE is a source-agnostic post-hoc correction for LLM unlearning updates that suppresses components aligned with high-energy retained activation directions while preserving the forgetting carrier.

  51. CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models

    cs.LG 2026-06 conditional novelty 7.0

    CheckMIABench converts LLMs with intermediate checkpoints into clean MIA testbeds by using pre- and post-checkpoint training data from the same distribution and evaluates published attacks on Pythia and OLMo models wh...

  52. How Linear Is a Transformer Feed-Forward Block? Per-Block Linear Recoverability Is Learned, Not Architectural

    cs.LG 2026-06 unverdicted novelty 7.0

    Linear recoverability of transformer FFN blocks varies widely across depth, is learned during training, and is independent of the activation function.

  53. LongSpike: Fractional Order Spiking State Space Models for Efficient Long Sequence Learning

    cs.LG 2026-06 unverdicted novelty 7.0

    LongSpike integrates fractional-order state-space modeling into spiking neural networks, enabling better long-sequence performance than prior SNNs on LRA, WikiText-103, and Speech Commands benchmarks while retaining s...

  54. Representing Time Series as Structured Programs for LLM Reasoning

    cs.LG 2026-06 unverdicted novelty 7.0

    T2SP converts time series into structured programs for trends, periods, and events, enabling off-the-shelf LLMs to perform better on editing, captioning, and QA tasks than raw string inputs.

  55. Boltzmann Attention: Learnable Ising Couplings for Cooperative Attention

    cs.LG 2026-06 unverdicted novelty 7.0

    Boltzmann attention augments query-key attention with learnable Ising pairwise couplings, yielding consistent gains over softmax attention on character language modeling and bracket matching that increase with sequenc...

  56. Quo Vadis, Visual In-Context Learning? A Unified Benchmark Across Domains and Tasks

    cs.CV 2026-06 unverdicted novelty 7.0

    The paper constructs the VIBE benchmark and evaluates six visual in-context learning models on 14 datasets, 12 tasks, and 106 combinations under a unified one-shot protocol, revealing limitations and failure modes.

  57. Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks

    cs.CV 2026-06 unverdicted novelty 7.0

    Earth-OneVision is a unified 2B-parameter RS-MLLM supporting six modalities and nine tasks via FGVLA, SLIS, and PCMA mechanisms plus a 34M QA-pair dataset, reporting competitive or superior benchmark results versus la...

  58. In Defense of Information Leakage in Concept-based Models

    cs.LG 2026-06 conditional novelty 7.0

    Concept-based models can use controlled 'benign' information leakage to remain accurate and intervenable under real-world concept incompleteness by reframing their training objective.

  59. Beyond Absolute Imitation: Anchored Residual Guidance for Privileged On-Policy Distillation

    cs.LG 2026-06 unverdicted novelty 7.0

    AR-OPD disentangles privileged supervision via anchored residual guidance to reduce hindsight leakage in on-policy distillation, reporting gains of 2.3 points over full privileged OPD and 7.9 over SFT on reasoning tasks.

  60. Alignment Defends LLMs from Property Inference Attacks

    cs.LG 2026-06 unverdicted novelty 7.0

    Alignment defenses adapted from DPO and GRPO mitigate property inference attacks on LLMs while preserving utility.