Pith. sign in

REVIEW 2 major objections 1 minor 60 cited by

Fully open post-training on Llama 3.1 bases yields models that surpass several closed systems on benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-05-11 05:03 UTC

load-bearing objection Tulu 3 stands out because it ships the full open post-training stack—data, code, decontamination pipeline, and RLVR—rather than just another set of benchmark numbers. the 2 major comments →

arxiv 2411.15124 v5 submitted 2024-11-22 cs.CL

Tulu 3: Pushing Frontiers in Open Language Model Post-Training

Nathan Lambert , Jacob Morrison , Valentina Pyatkin , Shengyi Huang , Hamish Ivison , Faeze Brahman , Lester James V. Miranda , Alisa Liu
show 15 more authors
Nouha Dziri Shane Lyu Yuling Gu Saumya Malik Victoria Graf Jena D. Hwang Jiangjiang Yang Ronan Le Bras Oyvind Tafjord Chris Wilhelm Luca Soldaini Noah A. Smith Yizhong Wang Pradeep Dasigi Hannaneh Hajishirzi
This is my paper
classification cs.CL
keywords language model post-trainingsupervised fine-tuningdirect preference optimizationreinforcement learningopen source modelsbenchmark evaluationdata decontamination
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Tulu 3, a family of models refined from Llama 3.1 bases through supervised finetuning, direct preference optimization, and a new method called reinforcement learning with verifiable rewards. These models achieve higher scores than the official instruct versions of Llama 3.1, Qwen 2.5, and Mistral, as well as closed models including GPT-4o-mini and Claude 3.5-Haiku. The work supplies complete datasets, code, infrastructure, and a multi-task evaluation scheme that includes development and unseen splits along with decontamination of training data. A sympathetic reader would care because post-training steps have long remained opaque, and an open recipe that reaches competitive performance removes a major barrier to further progress. The authors also report which training approaches failed to deliver reliable gains.

Core claim

Tulu 3 demonstrates that applying supervised finetuning, direct preference optimization, and reinforcement learning with verifiable rewards to Llama 3.1 base models, using carefully curated and decontaminated data, produces results that exceed those of Llama 3.1 instruct models, Qwen 2.5 instruct, Mistral instruct, GPT-4o-mini, and Claude 3.5-Haiku on the multi-task benchmarks.

What carries the argument

Reinforcement Learning with Verifiable Rewards (RLVR), which uses automatically verifiable signals to guide reinforcement learning instead of relying only on preference data or model judges.

Load-bearing premise

The multi-task evaluation scheme with decontamination and unseen splits accurately measures real generalization instead of overfitting to known benchmark distributions.

What would settle it

Running the released Tulu 3 models on a new collection of tasks assembled after the training data cutoff or on live user queries that shows no performance edge over the original instruct baselines.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Post-training can be fully reproduced and adapted to new domains using the released data, code, and procedures.
  • A combination of SFT, DPO, and RLVR reliably improves over base models on the tested benchmarks.
  • Decontamination and separate unseen splits provide a stricter test than standard benchmark reporting.
  • Some common training techniques do not produce consistent improvements and can be deprioritized.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The open release of the full pipeline could allow independent groups to match or exceed current closed-model performance on similar tasks.
  • RLVR may extend naturally to any domain where correctness can be checked automatically, such as code generation or mathematical reasoning.
  • Widespread adoption of the decontamination and multi-split evaluation approach could raise the bar for future post-training papers.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper introduces Tulu 3, a family of fully open post-trained models built on Llama 3.1 base models. It applies supervised fine-tuning (SFT), Direct Preference Optimization (DPO), and a novel Reinforcement Learning with Verifiable Rewards (RLVR) method, claiming superior performance over Llama 3.1 instruct, Qwen 2.5, Mistral, GPT-4o-mini, and Claude 3.5-Haiku. The work releases all training data, code, recipes, and models, while introducing a multi-task evaluation protocol with development/unseen splits and substantial decontamination of open benchmark datasets.

Significance. If the benchmark results hold under rigorous decontamination, the primary contribution is the complete, reproducible open recipe for modern post-training that includes both established methods and RLVR, plus analysis of approaches that failed to improve performance. Releasing the full data, code, infrastructure, and detailed report enables independent verification and adaptation, which is a substantial advance for the open-source community.

major comments (2)
  1. [Abstract / Evaluation section] Abstract and evaluation description: the claim of surpassing closed models rests on benchmark results after 'substantial decontamination,' yet no concrete method is specified (e.g., n-gram overlap thresholds, embedding similarity cutoffs, model-based detection, or paraphrase handling). Without these details, residual leakage on MMLU, GSM8K, or HumanEval cannot be ruled out, directly affecting the validity of the generalization claims.
  2. [Results] Results presentation: the abstract reports benchmark wins, but the manuscript must include full tables with per-task scores, error bars or multiple seeds, and explicit ablations isolating the contribution of RLVR versus SFT+DPO to substantiate the performance frontier claim.
minor comments (1)
  1. [Evaluation] The multi-task evaluation scheme with dev/unseen splits is a positive design choice; clarify how the unseen split is constructed and whether it overlaps with any training data beyond the stated decontamination.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We address each major comment below and will revise the manuscript to incorporate the suggested clarifications and additions.

read point-by-point responses

  1. Referee: [Abstract / Evaluation section] Abstract and evaluation description: the claim of surpassing closed models rests on benchmark results after 'substantial decontamination,' yet no concrete method is specified (e.g., n-gram overlap thresholds, embedding similarity cutoffs, model-based detection, or paraphrase handling). Without these details, residual leakage on MMLU, GSM8K, or HumanEval cannot be ruled out, directly affecting the validity of the generalization claims.

    Authors: We agree that explicit details on the decontamination procedure are necessary to support the generalization claims. In the revised manuscript, we will add a dedicated subsection in the evaluation protocol describing the exact decontamination methods, including the n-gram overlap thresholds applied, embedding similarity cutoffs, any model-based detection steps, and handling of paraphrases. This will allow readers to assess residual leakage risks on MMLU, GSM8K, HumanEval, and other benchmarks. revision: yes

  2. Referee: [Results] Results presentation: the abstract reports benchmark wins, but the manuscript must include full tables with per-task scores, error bars or multiple seeds, and explicit ablations isolating the contribution of RLVR versus SFT+DPO to substantiate the performance frontier claim.

    Authors: We will strengthen the results section to meet this requirement. The revised paper will include comprehensive tables with per-task scores across all evaluated benchmarks, report error bars or multi-seed averages where computationally feasible, and add explicit ablation experiments that isolate the contribution of RLVR relative to the SFT+DPO baseline. These changes will more clearly substantiate the performance claims and the value of the novel RLVR method. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical post-training results with released artifacts

full rationale

The paper reports experimental outcomes from SFT, DPO, and the introduced RLVR on Llama 3.1 bases, evaluated via multi-task benchmarks with decontamination and unseen splits. No derivation chain, equations, or first-principles predictions exist that could reduce to inputs by construction; performance claims rest on direct training runs and external verification via released data, code, and models rather than self-referential metrics or fitted parameters renamed as predictions. Self-citations to prior Tulu work are present but non-load-bearing for the central empirical claims, which remain independently falsifiable.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The central claims rest on standard supervised learning assumptions (i.i.d. data, gradient descent convergence) and benchmark validity; no new invented entities or ad-hoc axioms are introduced in the abstract. Free parameters are the usual training hyperparameters (learning rates, batch sizes, reward scales) whose specific values are not detailed here.

pith-pipeline@v0.9.0 · 5693 in / 1154 out tokens · 23646 ms · 2026-05-11T05:03:07.115505+00:00 · methodology

0 comments
read the original abstract

Language model post-training is applied to refine behaviors and unlock new skills across a wide range of recent language models, but open recipes for applying these techniques lag behind proprietary ones. The underlying training data and recipes for post-training are simultaneously the most important pieces of the puzzle and the portion with the least transparency. To bridge this gap, we introduce Tulu 3, a family of fully-open state-of-the-art post-trained models, alongside its data, code, and training recipes, serving as a comprehensive guide for modern post-training techniques. Tulu 3, which builds on Llama 3.1 base models, achieves results surpassing the instruct versions of Llama 3.1, Qwen 2.5, Mistral, and even closed models such as GPT-4o-mini and Claude 3.5-Haiku. The training algorithms for our models include supervised finetuning (SFT), Direct Preference Optimization (DPO), and a novel method we call Reinforcement Learning with Verifiable Rewards (RLVR). With Tulu 3, we introduce a multi-task evaluation scheme for post-training recipes with development and unseen evaluations, standard benchmark implementations, and substantial decontamination of existing open datasets on said benchmarks. We conclude with analysis and discussion of training methods that did not reliably improve performance. In addition to the Tulu 3 model weights and demo, we release the complete recipe -- including datasets for diverse core skills, a robust toolkit for data curation and evaluation, the training code and infrastructure, and, most importantly, a detailed report for reproducing and further adapting the Tulu 3 approach to more domains.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. DataComp-VLM: Improved Open Datasets for Vision-Language Models

    cs.CV 2026-06 conditional novelty 8.0

    DataComp-VLM benchmark shows instruction-heavy data mixing outperforms filtering for VLM training, with DCVLM-Baseline achieving 63.6% on 33 tasks for 8B models (+5.4pp over FineVision).

  2. A Pre-Registered Causal Partition of Self-Consistency Elicitation and Reward Design in RLVR

    cs.AI 2026-06 conditional novelty 8.0

    Derives an exact telescoping decomposition of the naive RLVR reward-design estimator into null, elicitation, and reward-design terms on a tabular-GRPO simulator, measures the components across prior strengths, and val...

  3. Masked Diffusion Decoding as $x$-Prediction Flow

    cs.CL 2026-06 unverdicted novelty 7.0

    Masked diffusion LMs can use continuous x-prediction flow with token-wise asynchronous updates and an RL policy network to reach 97% performance on HumanEval using only 25% of the usual decoding budget.

  4. Training for the Model You Return: Improving Optimization for Iterate-Averaged Language Models

    cs.LG 2026-06 unverdicted novelty 7.0

    PACE is a clipped per-coordinate controller added to AdamW that improves the limiting error of the returned iterate average in both quadratic analysis and LM experiments.

  5. Training for the Model You Return: Improving Optimization for Iterate-Averaged Language Models

    cs.LG 2026-06 unverdicted novelty 7.0

    PACE is an AdamW wrapper derived from optimal control that improves the limiting error of the returned exponential-moving-average model in both theory and LM experiments.

  6. A Verifiable Search Is Not a Learnable Chain-of-Thought

    cs.LG 2026-06 unverdicted novelty 7.0

    Verifiable search procedures cannot be learned as forward chain-of-thought by language models; they instead learn memorization, verification, or require precomputed catalogs.

  7. Dual Dimensionality for Local and Global Attention

    cs.CL 2026-06 unverdicted novelty 7.0

    Distance-Adaptive Representation (DAR) keeps full KV dimensionality inside a local window and reduces it to 1/4 outside, matching full-dimensional baselines on pretraining (70M-410M) and 1B-scale fine-tuning while uni...

  8. Forecasting Future Behavior as a Learning Task

    cs.AI 2026-06 unverdicted novelty 7.0

    Behavior Forecasters trained on LRM trajectories outperform larger models in predicting repeatability and input sensitivity at low cost.

  9. Bittensor Agent Arenas as a Trajectory Primitive: Distilling a Shopping Agent from ShoppingBench Subnet Traces

    cs.LG 2026-06 unverdicted novelty 7.0

    Trajectories from a Bittensor ShoppingBench subnet arena, filtered to retain only agentic tool-calling behavior, enable SFT+GRPO post-training of Qwen3-4B to 42.7% ASR on leak-guarded held-out tests, nearly matching s...

  10. CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning

    cs.CV 2026-06 unverdicted novelty 7.0

    CapRL++ applies reinforcement learning with verifiable rewards to dense image and video captioning by scoring captions via the accuracy of a vision-free LLM answering MCQs from the caption alone.

  11. The Hidden Bias of Process Reward Models:PRISM for Rewarding the Right Reasoning

    cs.LG 2026-06 unverdicted novelty 7.0

    PRISM is a contrastive, policy-aware training framework for process reward models that reduces false positives by 22% on PRMBench and boosts downstream accuracy up to 33% in Best-of-N selection by learning reliable re...

  12. HARP: Efficient Data Selection for Finetuning Large Language Models

    cs.LG 2026-06 unverdicted novelty 7.0

    HARP is a train-based data selector for LLM finetuning that uses hierarchical active region pruning and empirical Bayes posteriors to achieve up to 8.9 point gains with roughly 7 times fewer training examples.

  13. LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding

    cs.CL 2026-06 unverdicted novelty 7.0

    LazyAttention kernelizes deferred positional encoding to enable zero-copy, position-agnostic KV cache reuse, delivering 1.37× lower TTFT and 1.40× higher throughput than Block-Attention under skewed document distribut...

  14. RogueMerge: Robust and Unified Attacks against LLM Model Merging

    cs.CR 2026-06 unverdicted novelty 7.0

    RogueMerge is a unified attack method that jointly optimizes task vectors to succeed after merging, using stochastic min-max simulation for unknown merging settings and a Taylor-approximated DRO for prompt generalizat...

  15. MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models

    cs.CL 2026-05 unverdicted novelty 7.0

    MENTIS applies layerwise covariance torsion (T1), spectral torsion (T2), and ERA localization to paired IT/PA 7-8B models, finding selective larger shifts for normative concepts, negative correlation with entropy, and...

  16. How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions

    cs.SE 2026-05 unverdicted novelty 7.0

    Observational study of 20,574 sessions identifies seven misalignment forms where 90.5% cause effort/trust costs and 91.5% require explicit user correction, varying by interface and over time.

  17. Cybersecurity AI (CAI) Dataset

    cs.CR 2026-05 unverdicted novelty 7.0

    CAI Dataset is presented as the largest described corpus of LLM-driven hacker trajectories, with the claim that operator data concentration in frontier-model providers creates a major security risk best addressed by o...

  18. Explicit Critic Guidance for Aligning Diffusion Models

    cs.LG 2026-05 unverdicted novelty 7.0

    Introduces a state-aligned latent actor-critic framework that lets diffusion models act as their own timestep-conditioned value functions for trajectory-level RL post-training and inference steering.

  19. Extracting Training Data from Diffusion Language Models via Infilling

    cs.CL 2026-05 unverdicted novelty 7.0

    Infilling extraction on diffusion language models extracts up to three times more verbatim sequences than prefix methods and achieves higher recall on redacted emails than autoregressive models.

  20. RECIPE: Procedural Planning via Grounding in Instructional Video

    cs.CV 2026-05 unverdicted novelty 7.0

    RECIPE improves visual procedural planners by rewarding plans according to their grounding quality in ASR transcripts via GRPO, yielding +7–8 in-domain and up to +16 zero-shot macro-accuracy gains over base models and...

  21. A Data-Efficient Path to Multilingual LLMs: Language Expansion via Post-training PARAM$\Delta$ Integration into Upcycled MoE

    cs.CL 2026-05 unverdicted novelty 7.0

    PARAMΔ upcycles dense models to MoE for per-language experts and grafts post-training deltas to enable data-efficient language expansion while preserving original capabilities.

  22. Reasoning Portability: Guiding Continual Learning for MLLMs in the RLVR Era

    cs.LG 2026-05 unverdicted novelty 7.0

    Formalizes Reasoning Portability (RP) and proposes RDB-CL to modulate per-sample KL regularization in RLVR for MLLM continual learning, achieving +12.0% Last accuracy over vanilla RLVR baseline by preserving reusable ...

  23. MeMo: Memory as a Model

    cs.CL 2026-05 unverdicted novelty 7.0

    MeMo encodes new knowledge into a separate memory model for frozen LLMs, achieving strong performance on BrowseComp-Plus, NarrativeQA, and MuSiQue while capturing cross-document relationships and remaining robust to r...

  24. Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Sampling

    cs.LG 2026-05 unverdicted novelty 7.0

    DRATS derives a minimax objective from a feasibility formulation of MTRL to adaptively sample tasks with the largest return gaps, leading to better worst-task performance on MetaWorld benchmarks.

  25. CurveBench: A Benchmark for Exact Topological Reasoning over Nested Jordan Curves

    cs.CV 2026-05 unverdicted novelty 7.0

    CurveBench is a new benchmark for recovering rooted containment trees from images of nested Jordan curves, where the strongest model reaches only 19.1% accuracy on hard cases and fine-tuning lifts an open model to 33....

  26. CurveBench: A Benchmark for Exact Topological Reasoning over Nested Jordan Curves

    cs.CV 2026-05 unverdicted novelty 7.0

    CurveBench benchmark reveals that even leading VLMs like Gemini 3.1 Pro reach only 71.1% accuracy recovering containment trees on easy nested-curve images and 19.1% on hard versions, while fine-tuning lifts an open 8B...

  27. Learning, Fast and Slow: Towards LLMs That Adapt Continually

    cs.LG 2026-05 unverdicted novelty 7.0

    Fast-Slow Training uses context optimization as fast weights alongside parameter updates as slow weights to achieve up to 3x better sample efficiency, higher performance, and less catastrophic forgetting than standard...

  28. No More, No Less: Task Alignment in Terminal Agents

    cs.LG 2026-05 unverdicted novelty 7.0

    The TAB benchmark reveals that frontier terminal agents achieve high task completion but low selective alignment with relevant environmental cues over distractors, and prompt-injection defenses block both.

  29. A Recipe for Long-Context Reasoning in Large Language Models via On-Policy Optimization and Distillation

    cs.CL 2026-05 unverdicted novelty 7.0

    dGRPO merges outcome-based policy optimization with dense teacher guidance from on-policy distillation, yielding more stable long-context reasoning on the new LongBlocks synthetic dataset.

  30. Variance-aware Reward Modeling with Anchor Guidance

    stat.ML 2026-05 unverdicted novelty 7.0

    Anchor-guided variance-aware reward modeling uses two response-level anchors to resolve non-identifiability in Gaussian models of pluralistic preferences, yielding provable identification, a joint training objective, ...

  31. K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs

    cs.CL 2026-05 conditional novelty 7.0

    K12-KGraph is a textbook-derived knowledge graph that powers a new benchmark revealing LLMs' poor curriculum cognition and a small training corpus that outperforms general instruction data on educational tasks.

  32. KL for a KL: On-Policy Distillation with Control Variate Baseline

    cs.LG 2026-05 unverdicted novelty 7.0

    vOPD stabilizes on-policy distillation gradients by subtracting a closed-form per-token negative reverse KL baseline as a detached control variate, preserving unbiasedness while lowering variance and matching expensiv...

  33. Beyond Negative Rollouts: Positive-Only Policy Optimization with Implicit Negative Gradients

    cs.CL 2026-05 unverdicted novelty 7.0

    POPO uses bounded importance sampling on positive rollouts and a siamese policy network to achieve implicit negative gradients and stable optimization, matching or exceeding GRPO on math benchmarks such as 36.67% on A...

  34. MolmoAct2: Action Reasoning Models for Real-world Deployment

    cs.RO 2026-05 unverdicted novelty 7.0

    MolmoAct2 delivers an open VLA model with new specialized components, datasets, and techniques that outperforms baselines on benchmarks while releasing all weights, code, and data for real-world robot use.

  35. Reference-Sampled Boltzmann Projection for KL-Regularized RLVR: Target-Matched Weighted SFT, Finite One-Shot Gaps, and Policy Mirror Descent

    cs.LG 2026-05 unverdicted novelty 7.0

    Reference-sampled weighted SFT with prompt-normalized Boltzmann weights induces the same policy as fixed-reference KL-regularized RLVR, with BOLT as the estimator and a finite one-shot error decomposition separating c...

  36. Parallel-SFT: Improving Zero-Shot Cross-Programming-Language Transfer for Code RL

    cs.CL 2026-04 unverdicted novelty 7.0

    Parallel-SFT mixes parallel programs across languages during SFT to produce more transferable RL initializations, yielding better zero-shot generalization to unseen programming languages.

  37. Multi-modal Reasoning with LLMs for Visual Semantic Arithmetic

    cs.AI 2026-04 unverdicted novelty 7.0

    SAri-RFT applies GRPO-based reinforcement fine-tuning to LVLMs on novel two-term and three-term visual semantic arithmetic tasks, reaching SOTA on the new IRPD dataset and Visual7W-Telling.

  38. Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning

    cs.CL 2026-04 unverdicted novelty 7.0

    CoT-PoT ensembling achieves self-consistency accuracy in LLMs with only two samples for 78.6% of tasks, reducing computation by 9.3x compared to standard methods.

  39. You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass

    cs.CV 2026-04 unverdicted novelty 7.0

    A multi-response discriminative reward model scores N candidates in one pass via concatenation and cross-entropy, achieving SOTA on multimodal benchmarks and improving RL policies over single-response baselines.

  40. SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructions

    cs.AI 2026-04 unverdicted novelty 7.0

    SUPERNOVA adapts instruction-tuning data for RLVR and achieves up to 52.8% relative gains on general reasoning benchmarks like BBEH through targeted task selection and mixing.

  41. ALTO: Adaptive LoRA Tuning and Orchestration for Heterogeneous LoRA Training Workloads

    cs.LG 2026-04 unverdicted novelty 7.0

    ALTO accelerates LoRA tuning up to 13.8x by monitoring loss trajectories for early stopping, using fused grouped GEMM with rank-local adapter parallelism, and combining intra- and inter-task scheduling for heterogeneo...

  42. What If Consensus Lies? Selective-Complementary Reinforcement Learning at Test Time

    cs.LG 2026-03 unverdicted novelty 7.0

    SCRL adds selective positive pseudo-labeling and entropy-gated negative pseudo-labeling to test-time RL, reducing noise from weak consensus and improving LLM reasoning on benchmarks.

  43. Beyond Variance: Prompt-Efficient RLVR via Rare-Event Amplification and Bidirectional Pairing

    cs.LG 2026-02 unverdicted novelty 7.0

    Positive-negative prompt pairing with weighted GRPO improves RLVR sample efficiency, raising AIME 2025 Pass@8 from 16.8 to 22.2 on Qwen2.5-Math-7B while matching large-scale training.

  44. HetRL: Efficient Reinforcement Learning for LLMs in Heterogeneous Environments

    cs.DC 2025-12 unverdicted novelty 7.0

    HetRL delivers up to 9.17x higher throughput for LLM RL training on heterogeneous GPUs by using hybrid and ILP-based schedulers to solve a joint optimization problem over computation and data dependencies.

  45. ProfBench: Multi-Domain Rubrics requiring Professional Knowledge to Answer and Judge

    cs.CL 2025-10 unverdicted novelty 7.0

    ProfBench is a new multi-domain benchmark with human-expert rubrics for judging LLM responses on professional tasks, showing top models reach only 65.9% performance while providing cheap LLM judges that reduce evaluat...

  46. Decision Potential Surface: A Theoretical and Practical Approximation of Large Language Model Decision Boundary

    cs.LG 2025-09 unverdicted novelty 7.0

    Defines Decision Potential Surface (DPS) whose zero isohypse equals an LLM decision boundary and supplies a K-sample approximation algorithm with derived upper bounds on absolute, expected, and concentration errors.

  47. Task-Dependent Evaluation of LLM Output Homogenization: A Taxonomy-Guided Framework

    cs.CL 2025-09 conditional novelty 7.0

    Proposes a task taxonomy for functional diversity in LLM outputs, validates it via user study, introduces targeted sampling to boost diversity only where needed, and presents evidence that the diversity-quality tradeo...

  48. Token Buncher: Shielding LLMs from Harmful Reinforcement Learning Fine-Tuning

    cs.LG 2025-08 unverdicted novelty 7.0

    TokenBuncher constrains response entropy via entropy-as-reward RL and a Token Noiser to stop harmful RL fine-tuning while keeping benign performance intact.

  49. Reinforcement Learning for Reasoning in Large Language Models with One Training Example

    cs.LG 2025-04 accept novelty 7.0

    One training example via RLVR boosts LLM math reasoning from 17.6% to 35.7% average across six benchmarks.

  50. Active-GRPO: Adaptive Imitation and Self-Improving Reasoning for Molecular Optimization

    cs.LG 2026-07 unverdicted novelty 6.0

    Active-GRPO reaches 0.1773 average SRxSim on TOMG-Bench MOLOPT by adaptively switching between imitation and self-reinforcement while upgrading references, outperforming GRPO and RePO.

  51. DataComp-VLM: Improved Open Datasets for Vision-Language Models

    cs.CV 2026-06 unverdicted novelty 6.0

    DataComp-VLM benchmark shows instruction-heavy data mixtures outperform caption-heavy ones for VLM training, with DCVLM-Baseline reaching 63.6% on 33 tasks using 200B tokens, +5.4pp over FineVision.

  52. Reinforcement Learning without Ground-Truth Solutions can Improve LLMs

    cs.LG 2026-06 unverdicted novelty 6.0

    RiVER applies calibrated ranking rewards from execution scores to train LLMs on score-based tasks without ground-truth, producing gains on both heuristic contests and exact-solution coding benchmarks.

  53. E-TTS: A New Embodied Test-Time Scaling Framework for Robotic Manipulation

    cs.RO 2026-06 unverdicted novelty 6.0

    E-TTS introduces a plug-and-play test-time scaling method for embodied tasks that unifies reasoning-action sampling with history buffers and closed-loop refinement to improve performance on manipulation benchmarks.

  54. MiniOpt: Reasoning to Model and Solve General Optimization Problems with Limited Resources

    cs.LG 2026-06 unverdicted novelty 6.0

    MiniOpt is an RL framework that decomposes optimization reasoning into modeling and solver generation, achieving top solving accuracy for models under 10B parameters across diverse problem types using OptReward and op...

  55. CALIBER: Calibrating Confidence Before and After Reasoning in Language Models

    cs.CL 2026-06 unverdicted novelty 6.0

    CALIBER elicits and supervises pre-reasoning confidence with prompt-level success probability and post-reasoning confidence with answer-level correctness, cutting ECE by 52.5% on BigMathDigits for a 7B model while rem...

  56. What are Key Factors for Updates in RL for LLM Reasoning?

    cs.CL 2026-06 unverdicted novelty 6.0

    Theoretical analysis of RLVR update dynamics leads to ACPO, an adaptive clipping method that outperforms DAPO and CISPO on reasoning benchmarks with 3B and 7B models.

  57. Decodable but Not Faithful: Coupling Natural-Language Rationales to Programmatic Verifiers

    cs.LG 2026-06 unverdicted novelty 6.0

    Consistency training decodes verifier information from rationale representations but does not produce faithful natural-language explanations.

  58. VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Termination

    cs.CL 2026-06 unverdicted novelty 6.0

    VoidPadding decouples padding from termination in MDLMs via a new [VOID] token, delivering +17.84 average benchmark points and 55.7% fewer decoding steps on Dream-7B-Instruct.

  59. SKILL.nb: Selective Formalization and Gated Execution for Durable Agent Workflows

    cs.AI 2026-06 unverdicted novelty 6.0

    SKILL.nb uses selective formalization and gate-conditioned execution in auditable notebooks to improve durability of agent workflows, achieving 53.7% success on WebArena-Verified with 91.7% retention across re-executions.

  60. The Reliability Gap in Benchmark Auditing: Distribution Shift and Scale as Failure Modes of Contamination Detection

    cs.AI 2026-06 conditional novelty 6.0

    Systematic tests of LLM contamination detectors across 27 models show frequent failures from distribution shift and scale, concluding statistical methods cannot replace transparent data provenance.

Reference graph

Works this paper leans on

45 extracted references · 45 canonical work pages · cited by 181 Pith papers · 2 internal anchors

  1. [1]

    URL https://openreview.net/forum?id=Ep0TtjVoap. D. Groeneveld, I. Beltagy, P. Walsh, A. Bhagia, R. Kinney, O. Tafjord, A. Jha, H. Ivison, I. Magnusson, Y. Wang, S. Arora, D. Atkinson, R. Authur, K. R. Chandu, A. Cohan, J. Dumas, Y. Elazar, Y. Gu, J. Hessel, T. Khot, W. Merrill, J. D. Morrison, N. Muennighoff, A. Naik, C. Nam, M. E. Peters, V. Pyatkin, A. ...

    work page internal anchor Pith review arXiv 2024

  2. [2]

    RoBERTa: A Robustly Optimized BERT Pretraining Approach

    URL https://openreview.net/forum?id=1qvx610Cu7. Y. Liu. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692 , 364, 2019. S. Longpre, L. Hou, T. Vu, A. Webson, H. W. Chung, Y. Tay, D. Zhou, Q. V. Le, B. Zoph, J. Wei, et al. The flan collection: Designing data and methods for effective instruction tuning.arXiv preprint ar...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.7717/peerj-cs.103 1907

  3. [3]

    {% "{% "{{␣’<|system|>\n’␣+␣message [ ’ content ’]␣+␣ ’\n’␣}}

    Association for Computational Linguistics. URLhttps://aclanthology.org/2024.emnlp-main.79. C. Xu, Q. Sun, K. Zheng, X. Geng, P. Zhao, J. Feng, C. Tao, and D. Jiang. Wizardlm: Empowering large language models to follow complex instructions.arXiv preprint arXiv:2304.12244 , 2023. 58 H. Xu, B. Liu, L. Shu, and P. Yu. BERT post-training for review reading com...

    work page doi:10.18653/v1/n19-1242 2024

  4. [4]

    The above example is not tied to any particular persona, but you should create one that is unique and specific to the given persona

    work page

  5. [5]

    The instruction should contain all the following verifiable constraint(s):{constraints}

    work page

  6. [6]

    User instruction:

    Your output should start with "User instruction:". Your output should not include an answer to the instruction. Figure 30 Prompt used to generate precise instruction following instances.{persona} are borrowed from Chan et al. (2024). We use the set of{constraints} defined in Zhou et al. (2023). Example seeds are manually written by authors for each constr...

    work page 2024

  7. [7]

    You should rewrite the instruction coherently while relaxing one of the following con- straint categories:{constraints}

    work page

  8. [8]

    Remember to entirely relax one of the constraint category

    work page

  9. [9]

    User instruction:

    Your output should start with "User instruction:". Your output should not include an answer to the instruction. Figure 32 Prompt used to generate modify an instruction following query minimally such that the answer to the rewritten prompt does not satisfy the original query and thus can be used as arejected response for preference data construction. Hard ...

    work page

  10. [10]

    Only top talents can solve it correctly

    The math problem should be challenging and involve advanced mathematical skills and knowledge. Only top talents can solve it correctly

    work page

  11. [11]

    You should make full use of the persona description to create the math problem to ensure that the math problem is unique and specific to the persona

    work page

  12. [12]

    Math problem:

    Your response should always start with "Math problem:". Your response should not include a solution to the created math problem

    work page

  13. [13]

    Figure 33 Prompt used to generate hard math word problems.{persona} are borrowed from Chan et al

    Your created math problem should include no more than 2 sub-problems. Figure 33 Prompt used to generate hard math word problems.{persona} are borrowed from Chan et al. (2024). 70 Hard Math Problems (response) Provide solution to the given math problem. Problem: {generated_math_problem} Note: Provide your solution step-by-step, and end your solution in a n...

    work page 2024

  14. [14]

    Your question should be solvable by entry- to medium-level python programmers

    work page

  15. [15]

    Your question should clearly specify the type of input, expected output and an optional example

    work page

  16. [16]

    Question: Write a python function to

    Your response should always start with "Question: Write a python function to"

    work page

  17. [17]

    Figure 35 Prompt used to generate code completion instances.{persona} are borrowed from Chan et al

    Your response should not include a solution to the created coding problem. Figure 35 Prompt used to generate code completion instances.{persona} are borrowed from Chan et al. (2024). Code Completion (response) Provide solution to the given python programming question. Question: {generated_code_problem} Note:

    work page 2024

  18. [18]

    Your response should always start with the function definition and end with the final re- turn statement

    work page

  19. [19]

    Instruction

    Your response should only and only include python function. Figure 36 Prompt used to generate code completion. 71 System prompt for LLM-as-a-judge Your role is to evaluate text quality based on given criteria. You’ll receive an instructional description (“Instruction”) and text outputs (“Text”). Understand and interpret instructions to evaluate effectivel...

    work page 2023

  20. [20]

    Irrelevant: No alignment

    work page

  21. [21]

    Partial Focus: Addresses one aspect poorly

    work page

  22. [22]

    - (2) Acknowledges both but slight deviations

    Partial Compliance: - (1) Meets goal or restrictions, neglecting other. - (2) Acknowledges both but slight deviations

    work page

  23. [23]

    Almost There: Near alignment, minor deviations

    work page

  24. [24]

    Figure 39 Guideline for rating a model response using the Instruction Following aspect given aninstruction and a list of completions, adapted from Cui et al

    Comprehensive Compliance: Fully aligns, meets all requirements. Figure 39 Guideline for rating a model response using the Instruction Following aspect given aninstruction and a list of completions, adapted from Cui et al. (2023). 73 Informativeness or Helpfulness Aspect (prompt) # Informativeness / Helpfulness Assessment Evaluate if model’s outputs fulfil...

    work page 2023

  25. [25]

    Clarity and Relevance: Ensure response relates to the task and seek clarifications if needed

    work page

  26. [26]

    Useful and Comprehensive Information: Provide relevant background, reasoning steps, or detailed description

    work page

  27. [27]

    Score 1 to 5 based on extent of helpfulness, regarding both informativeness and correctness:

    Not Lengthy, No Repetition: Avoid verbosity or recycling content. Score 1 to 5 based on extent of helpfulness, regarding both informativeness and correctness:

    work page

  28. [28]

    Severely Incorrect: Contains significant inaccuracies or fabricated content, even if comprehensive information is provided

    work page

  29. [29]

    Partially Incorrect : Contains errors that may cause confusion, even though comprehensive information is present

    work page

  30. [30]

    Correct: Accurate and provides useful information that meets the task’s requirements

    work page

  31. [31]

    Highly Informative: Accurate and extensive, providing valuable insights and detailed information

    work page

  32. [32]

    Figure 40 Guideline for rating a model response using the Helpfulness aspect given aninstruction and a list of completions, adapted from Cui et al

    Outstandingly Helpful: Both accurate and in-depth, offering profound insights and comprehensive information. Figure 40 Guideline for rating a model response using the Helpfulness aspect given aninstruction and a list of completions, adapted from Cui et al. (2023). 74 Honesty Aspect (prompt) # Honesty and Uncertainty Expression Assessment Assess how well t...

    work page 2023

  33. [33]

    Weakeners: e.g., ‘I guess,’ ‘probably.’

    work page

  34. [34]

    - No uncertainty expression indicate confidence

    Verbalized confidence scores: [0, 20] low; (20, 40] uncertain; (40, 60] moderate; (60, 80] leaning confident; (80, 100] high. - No uncertainty expression indicate confidence. - Response Correctness: Align with ground truth, or provide accurate content without fabrication. Scoring: Rate outputs 1 to 5 (or “N/A”):

    work page

  35. [35]

    Confidently Incorrect: Confident but entirely wrong

    work page

  36. [36]

    - Unconfident and entirely wrong

    Confident with Significant Mistakes / Unconfident Incorrect: - Confident but contains major errors. - Unconfident and entirely wrong

    work page

  37. [37]

    - Confident but contains minor errors

    Uncertain / ‘I Don’t Know’ / Subtle Mistakes: - ‘I don’t know’ or declines. - Confident but contains minor errors. - Unconfident and contains significant mistakes

    work page

  38. [38]

    - Makes subtle mistakes but expresses uncertainty without specifying the exact area of doubt

    Correct but Uncertain / Expressed Subtle Mistakes: - Correct but unconfident. - Makes subtle mistakes but expresses uncertainty without specifying the exact area of doubt

    work page

  39. [39]

    - Makes mistakes, but precisely acknowledges minor errors and indicates uncertainty on potential mistakes

    Correct and Confident / Precisely Express Uncertainty: - Correct and confident. - Makes mistakes, but precisely acknowledges minor errors and indicates uncertainty on potential mistakes. N/A. Not Applicable: For creative writing tasks. Figure 41 Guideline for rating a model response using the Honesty aspect given aninstruction and a list of completions, a...

    work page 2023

  40. [40]

    Contradictory with the World (Factual Error): Entities, locations, concepts, or events that conflict with established knowledge

    work page

  41. [41]

    Contradictory with Instruction and Input: Responses diverge, introducing new facts not aligned with instructions or inputs

    work page

  42. [42]

    Scoring: Rate outputs 1 to 5 based on extent of hallucination:

    Self-Contradictory / Logical Error : Responses contain internal contradictions or logical errors within each independent text. Scoring: Rate outputs 1 to 5 based on extent of hallucination:

    work page

  43. [43]

    Completely Hallucinated: Entirely unreliable due to hallucinations

    work page

  44. [44]

    Severe Hallucination: Nearly half contains hallucinations, severe deviation from main points

    work page

  45. [45]

    Therefore, the answer is (ANSWER_LETTER)

    Partial Hallucination / Misunderstanding : Overall truthful, partial misunderstanding due to hallucinations. 4. Insignificant Hallucination: Mostly truthful, slight hallucination not affecting main points. 5. No Hallucination: Free of hallucinations. Figure 42 Guideline for rating a model response using the Truthfulness aspect given aninstruction and a li...

    work page 2023