REVIEW 2 major objections 2 minor 60 cited by
A 32B model finetuned on 1,000 reasoning traces and equipped with budget forcing exceeds o1-preview on competition math by up to 27%.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.3
2026-05-11 16:58 UTC
load-bearing objection A minimal fine-tune on 1k curated traces plus the 'Wait' append trick produces competitive test-time scaling on math benchmarks, but the mechanism still needs tighter controls to separate it from plain length extension. the 2 major comments →
s1: Simple test-time scaling
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Supervised finetuning on the s1K dataset of 1,000 curated reasoning traces combined with budget forcing during generation produces s1-32B, which exceeds o1-preview on competition math questions by up to 27% on MATH and AIME24; scaling the test-time budget with budget forcing further extrapolates performance beyond the base model, reaching 57% on AIME24 from a 50% starting point.
What carries the argument
Budget forcing, a method that terminates the model's output or appends 'Wait' multiple times when generation would otherwise end, to allocate more test-time compute and encourage double-checking of reasoning steps.
Load-bearing premise
That appending 'Wait' genuinely improves the substance and accuracy of the reasoning chain rather than simply lengthening outputs without adding corrective value.
What would settle it
Run the finetuned model on AIME24 or MATH problems while allowing unconstrained longer generation without any 'Wait' appends and compare accuracy to the budget-forced versions; if the accuracy gains disappear, the claim that budget forcing adds value beyond length fails.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces s1, a simple approach to test-time scaling for language models. It involves curating a 1,000-example dataset (s1K) of questions with high-quality reasoning traces selected for difficulty, diversity, and quality, validated via ablations. A technique called budget forcing is proposed to control test-time compute by either terminating the model's reasoning or appending the word 'Wait' to encourage further thinking and error correction. The Qwen2.5-32B-Instruct model is fine-tuned on s1K and combined with budget forcing to create s1-32B, which reportedly surpasses OpenAI's o1-preview on MATH and AIME24 benchmarks by up to 27%. Additionally, increasing the budget via more 'Wait' appends allows performance extrapolation on AIME24 from 50% to 57%. The work emphasizes simplicity and releases all components openly.
Significance. If the results hold under rigorous controls, this work is significant for providing an accessible, fully open-source demonstration of test-time scaling that achieves competitive or superior performance to closed models like o1-preview using only a small curated dataset and a lightweight inference intervention. Strengths include the emphasis on reproducibility (open model, data, code), ablations validating dataset criteria, and the empirical extrapolation result showing continued gains with increased test-time budget. It offers a practical baseline for the community studying reasoning in LLMs.
major comments (2)
- [Budget Forcing section] Budget Forcing section: The central claim that budget forcing enables effective test-time scaling rests on the assertion that appending 'Wait' 'often fixes incorrect reasoning steps.' However, the experiments lack a direct control comparing this mechanism to simply extending generation length via other means (e.g., raising the max token limit, using a generic 'continue' prompt, or sampling additional tokens without the specific intervention). Without isolating this, the reported gains (such as the 50% to 57% AIME24 extrapolation and the 27% lift over o1-preview) could be attributable to increased output length rather than a qualitative change in reasoning distribution. This is load-bearing for the paper's simplicity and effectiveness argument.
- [Main Results section] Main Results section: The claim that s1-32B exceeds o1-preview by up to 27% on MATH and AIME24 requires more detail on the exact evaluation protocol for the o1-preview baseline, including the number of samples or test-time compute budget allocated to it (given o1's undisclosed internal scaling). Additionally, reporting variance across multiple runs or statistical tests for the improvements would help assess robustness, especially since the abstract-only view leaves ambiguity on whether gains are consistent or sensitive to specific choices.
minor comments (2)
- [Abstract] Abstract: The phrase 'up to 27%' is used without specifying the exact benchmark and condition achieving the maximum; adding this precision would improve clarity.
- [Dataset Curation section] Dataset Curation section: While ablations on difficulty, diversity, and quality are mentioned, expanding the main text or appendix with quantitative metrics used to measure each criterion (e.g., diversity scores) would enhance reproducibility.
Simulated Author's Rebuttal
We thank the referee for their constructive and detailed comments. We address each major comment point by point below and outline the changes we will make to strengthen the manuscript.
read point-by-point responses
-
Referee: [Budget Forcing section] Budget Forcing section: The central claim that budget forcing enables effective test-time scaling rests on the assertion that appending 'Wait' 'often fixes incorrect reasoning steps.' However, the experiments lack a direct control comparing this mechanism to simply extending generation length via other means (e.g., raising the max token limit, using a generic 'continue' prompt, or sampling additional tokens without the specific intervention). Without isolating this, the reported gains (such as the 50% to 57% AIME24 extrapolation and the 27% lift over o1-preview) could be attributable to increased output length rather than a qualitative change in reasoning distribution. This is load-bearing for the paper's simplicity and effectiveness argument.
Authors: We agree that a direct comparison isolating the 'Wait' intervention from generic increases in output length is necessary to support the central claim. While our results show consistent gains from budget forcing, we acknowledge that the current experiments do not fully rule out length as the sole factor. In the revised manuscript, we will add new ablation experiments comparing budget forcing to (1) simply raising the maximum token limit without any prompt and (2) using a generic continuation prompt such as 'continue'. These results will be reported in an expanded Budget Forcing section, including quantitative comparisons on AIME24 and MATH to demonstrate whether the specific mechanism produces qualitatively different reasoning behavior beyond length alone. revision: yes
-
Referee: [Main Results section] Main Results section: The claim that s1-32B exceeds o1-preview by up to 27% on MATH and AIME24 requires more detail on the exact evaluation protocol for the o1-preview baseline, including the number of samples or test-time compute budget allocated to it (given o1's undisclosed internal scaling). Additionally, reporting variance across multiple runs or statistical tests for the improvements would help assess robustness, especially since the abstract-only view leaves ambiguity on whether gains are consistent or sensitive to specific choices.
Authors: We appreciate the request for greater transparency. For the o1-preview baseline, we accessed the model via the OpenAI API using its default configuration and standard test-time scaling behavior. Because o1-preview's internal scaling mechanisms are proprietary and undisclosed, we cannot report the precise compute budget allocated by OpenAI. In the revision, we will expand the Main Results section to fully document our evaluation protocol for both models, including API parameters (temperature, max tokens where applicable), the exact number of problems evaluated from each benchmark, and any sampling details. We will also explicitly note that s1-32B results are from single runs due to computational cost and will report performance consistency across varying budget-forcing levels as an indirect robustness check. Statistical tests will be added where multiple runs become feasible. revision: partial
Circularity Check
No significant circularity in empirical claims
full rationale
The paper reports an empirical pipeline: curation of s1K via ablations on difficulty/diversity/quality criteria, SFT of Qwen2.5-32B-Instruct, and application of budget forcing (append 'Wait' or terminate) whose effects are measured on MATH/AIME24. All performance numbers (e.g., 27% lift, 50% to 57% extrapolation) are observed outcomes, not quantities derived from the method definition or fitted parameters renamed as predictions. No equations, uniqueness theorems, or self-citations function as load-bearing premises that reduce the central result to its inputs by construction. The work is self-contained against external benchmarks with released data and code.
Axiom & Free-Parameter Ledger
free parameters (2)
- s1K dataset size =
1000
- Number of 'Wait' appends =
multiple
axioms (1)
- domain assumption The Qwen2.5-32B-Instruct base model is suitable for SFT on reasoning traces.
read the original abstract
Test-time scaling is a promising new approach to language modeling that uses extra test-time compute to improve performance. Recently, OpenAI's o1 model showed this capability but did not publicly share its methodology, leading to many replication efforts. We seek the simplest approach to achieve test-time scaling and strong reasoning performance. First, we curate a small dataset s1K of 1,000 questions paired with reasoning traces relying on three criteria we validate through ablations: difficulty, diversity, and quality. Second, we develop budget forcing to control test-time compute by forcefully terminating the model's thinking process or lengthening it by appending "Wait" multiple times to the model's generation when it tries to end. This can lead the model to double-check its answer, often fixing incorrect reasoning steps. After supervised finetuning the Qwen2.5-32B-Instruct language model on s1K and equipping it with budget forcing, our model s1-32B exceeds o1-preview on competition math questions by up to 27% (MATH and AIME24). Further, scaling s1-32B with budget forcing allows extrapolating beyond its performance without test-time intervention: from 50% to 57% on AIME24. Our model, data, and code are open-source at https://github.com/simplescaling/s1
Forward citations
Cited by 60 Pith papers
-
REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations
REALISTA optimizes continuous combinations of valid editing directions in latent space to produce realistic adversarial prompts that elicit hallucinations more effectively than prior methods, including on large reason...
-
The Coupling Tax: How Shared Token Budgets Undermine Visible Chain-of-Thought Under Fixed Output Limits
Shared token budgets between visible chain-of-thought and answers create a coupling tax that makes non-thinking competitive on math benchmarks, with a truncation decomposition predicting the crossover and split budget...
-
GIANTS: Generative Insight Anticipation from Scientific Literature
GIANTS-4B, trained with RL on a new 17k-example benchmark of parent-to-child paper insights, achieves 34% relative improvement over gemini-3-pro in LM-judge similarity and is rated higher-impact by a citation predictor.
-
RLCracker: Evaluating the Worst-Case Vulnerability of LLM Watermarks with Adaptive RL Attacks
RLCracker is a reinforcement learning attack that erases LLM watermarks at 98.5% success rate with minimal data and generalizes across ten schemes and multiple model sizes.
-
Does Verbose Chain-of-Thought Really Help? In-Distribution Evidence that Content, Not Length, Matters
In-distribution sampling across 25 models and controlled interventions with DAG-verified content show that semantic reasoning and validation content, not token count, drive CoT gains.
-
Convergence of Gradient Descent for General Neural Network Architectures Beyond the NTK Regime
Proves GD convergence to stationary point neighborhoods for general NN architectures beyond NTK via block-level analysis, analyticity, and local smoothness conditions.
-
A Verifiable Search Is Not a Learnable Chain-of-Thought
Verifiable search procedures cannot be learned as forward chain-of-thought by language models; they instead learn memorization, verification, or require precomputed catalogs.
-
KCSAT-ML: Probing Reasoning Models with Nationwide-Cohort Human Difficulty
KCSAT-ML benchmark supplies human error rates for math problems and DRG metric exposes that model accuracy collapses on high-human-error items while test-time scaling shows non-monotonic gains and alignment failures.
-
When to Think Deeply: Inhibitory Deliberation for LLM Reasoning
IDPR is a response-conditioned inhibitory deliberation method that trains a controller on fast-slow outcome pairs to decide when to override LLM fast answers, improving accuracy from 47.90% to 48.92% with slow reasoni...
-
Failed Reasoning Traces Tell You What Is Fixable (But Not by Reading Them)
Three problem-level trajectory features derived from the distributional signature of failed LLM rollouts enable failure clustering at 84.3% accuracy and a training-free routing rule that improves rescue by 12.2% on ha...
-
PANDO: Efficient Multimodal AI Agents via Online Skill Distillation
PANDO introduces an online skill-distillation method with a structured library, reflection, demotion, routing, compression, and cache-aware prompting that reaches 58.3% success on 910 VisualWebArena tasks using 58-61%...
-
PluRule: A Benchmark for Moderating Pluralistic Communities on Social Media
PluRule is a new multimodal multilingual benchmark showing that state-of-the-art vision-language models perform only marginally better than a trivial baseline at detecting specific rule violations in pluralistic onlin...
-
Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
Decoupling prefix source from token-level KL direction in autoregressive sequence KL yields four objectives unifying SFT, DAgger, offline RL and OPD, with KL mixing and entropy-gated curriculum improving math reasonin...
-
Uncovering the Representation Geometry of Minimal Cores in Overcomplete Reasoning Traces
Language models produce overcomplete reasoning traces where on average 46% of steps can be removed while preserving the answer in 86% of cases, with necessity concentrated in the top three steps.
-
Multi-Rollout On-Policy Distillation via Peer Successes and Failures
MOPD improves on-policy distillation for LLMs by using peer successes for positive patterns and failures for negative examples to create more informative teacher signals.
-
DUET: Optimize Token-Budget Allocation for Reinforcement Learning with Verifiable Rewards
DUET improves RLVR by allocating tokens across both prompt selection and rollout length, outperforming full-budget baselines even when using only half the tokens.
-
Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key
RL training on more expressive logical tasks follows a steeper power-law scaling with reasoning depth and transfers more efficiently to math and reasoning benchmarks.
-
Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key
RL training compute for logical reasoning follows a power law in proof depth whose exponent rises with logic expressiveness, and more expressive training yields larger gains on downstream benchmarks.
-
POSTCONDBENCH: Benchmarking Correctness and Completeness in Formal Postcondition Inference
POSTCONDBENCH is a new multilingual benchmark that evaluates LLM postcondition generation on real code using defect discrimination to assess completeness beyond surface matching.
-
VIDA: A dataset for Visually Dependent Ambiguity in Multimodal Machine Translation
VIDA provides 2,500 visually-dependent ambiguous translation examples and span-level disambiguation metrics; CoT-SFT on LVLMs improves out-of-distribution performance over standard SFT.
-
How Fast Should a Model Commit to Supervision? Training Reasoning Models on the Tsallis Loss Continuum
A single-parameter Tsallis loss continuum unifies SFT and RLVR, derives time-to-escape bounds for cold start, and yields GARL and PAFT estimators that improve performance on QA reasoning tasks.
-
Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning
A new SFT framework for MoE models combines bias-driven sparsification with gated condenser experts to retain long-tailed expert information, outperforming DenseMixer and ESFT by over 2.5% on math reasoning and common...
-
Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning
CoT-PoT ensembling achieves self-consistency accuracy in LLMs with only two samples for 78.6% of tasks, reducing computation by 9.3x compared to standard methods.
-
Towards Unconstrained Human-Object Interaction
Introduces the U-HOI task and shows MLLMs plus a language-to-graph pipeline can handle human-object interactions without any predefined vocabulary at training or inference time.
-
AI Achieves a Perfect LSAT Score
Language models achieve a perfect LSAT score, with experiments showing that internal thinking phases and a fine-tuned process reward model are key to high performance on logical reasoning questions.
-
Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
A single LLM improves its own reasoning by self-distilling from privileged verified traces as teacher to its question-only student policy, outperforming off-policy distillation and RL on math benchmarks with better to...
-
Reliable LLM-Based Edge-Cloud-Expert Cascades for Telecom Knowledge Systems
An edge-cloud-expert LLM cascade for telecom knowledge systems minimizes processing cost subject to misalignment-risk bounds via multiple hypothesis testing on knowledge and confidence scores.
-
PIAST: Rapid Prompting with In-context Augmentation for Scarce Training data
PIAST iteratively optimizes few-shot examples in prompts via Monte Carlo Shapley value estimation, outperforming prior automatic prompting methods and setting new SOTA on classification, simplification, and GSM8K with...
-
User-Assistant Bias in LLMs
LLMs show strong user bias in role-tagged contexts that is amplified by preference alignment and can be reduced or controlled through targeted fine-tuning and DPO.
-
Bayesian Social Deduction with Graph-Informed Language Models
Hybrid Bayesian-graph LLM agent reaches competitive performance against large models and achieves 67% win rate against humans in controlled Avalon play, outperforming baselines and human teammates.
-
Learning a Continue-Thinking Token for Enhanced Test-Time Scaling
A learned continue-thinking token, trained via RL on its embedding alone, improves math benchmark accuracy more than fixed-token budget forcing in a frozen language model.
-
DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning
DeepEyes uses reinforcement learning to teach vision-language models active perception and image-based thinking, yielding gains on perception, reasoning, grounding, and hallucination benchmarks.
-
L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning
LCPO trains L1 reasoning models to adhere to prompt-specified CoT lengths, supporting accuracy-compute trade-offs and yielding short reasoning models that outperform larger baselines at matched lengths.
-
When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors
LLMs exhibit data referencing errors across model sizes; a critic model detects them at 78.2% F1 and boosts accuracy up to 12% via filtering and rejection sampling.
-
Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
LOTUS uses a looped padded Transformer with parallel cross-entropy supervision on gold CoT tokens to match explicit CoT performance at 3B parameters while reducing thought-phase latency 2.5x-6.9x.
-
Addressing Over-Refusal in LLMs with Competing Rewards
SEAR trains one LLM via adversarial process rewards to explore harmful reasoning paths but flip to safe outputs, reducing over-refusal while preserving safety.
-
Curriculum Reinforcement Learning Can Incentivize Reasoning Capacity in LLMs Beyond the Base Model
Boundary-aware Curriculum RL raises average pass@256 by 9.8 points over base models and 10.3 points over vanilla RLVR on Qwen, Llama, and DeepSeek families.
-
Dynamic Rollout Editing for Reducing Overthinking in RL-Trained Reasoning Models
Dynamic Rollout Editing reduces overthinking in RL-trained LLMs by editing post-answer continuations in successful rollouts and preferring the edited versions within GRPO groups.
-
Reinforcing Dual-Path Reasoning in Spatial Vision Language Models
SR-REAL equips spatial VLMs with dual LOR and DTR reasoning paths trained via RL, achieving better benchmark performance through mutual reinforcement and generalization without per-task tuning.
-
MARD: Mirror-Augmented Reasoning Distillation for Mechanism-Level Drug-Drug Interaction Prediction
MARD-7B outperforms baselines and GPT-4o on novel drug pairs for mechanism-level DDI prediction via a new distillation pipeline with verifiable process rewards and releases all resources.
-
Back on Track: Aligning Rewards and States for Reasoning in Diffusion Large Language Models
PAPO improves reasoning performance in diffusion LLMs by converting sparse terminal rewards into dense step-wise credit and replaying real high-uncertainty trajectories, reporting gains up to 42.2% on Countdown.
-
Geometric Latent Reasoning Induces Shorter Generations in LLMs
GLR formulates latent reasoning as geometric path approximation in pretrained embedding space and reports shorter LLM generations on math tasks without an explicit length penalty.
-
Agent Explorative Policy Optimization for Multimodal Agentic Reasoning
AXPO addresses the Thinking-Acting Gap in agentic RL training by targeted resampling of tool calls in all-wrong subgroups, delivering +1.8pp gains over GRPO on nine multimodal benchmarks with an 8B model beating a 32B...
-
PathCal: State-Aware Reflection-Marker Calibration for Efficient Reasoning
PathCal calibrates reasoning paths by type-aware soft rebalancing of reflection-marker logits at uncertain states, yielding better efficiency-performance trade-offs on six benchmarks.
-
Taming the Thinker: Conditional Entropy Shaping for Adaptive LLM Reasoning
CES applies conditional bidirectional entropy control on top of DAPO to improve accuracy and shorten responses on mathematical benchmarks for 7B and 1.5B LLMs.
-
Are Rationales Necessary and Sufficient? Tuning LLMs for Explainable Misinformation Detection
LONSREX introduces a metric-based pipeline to identify necessary and sufficient rationales when creating training data for fine-tuning LLMs on explainable misinformation detection, addressing limitations of naive labe...
-
Self-Supervised On-Policy Distillation for Reasoning Language Models
SSOPD converts intra-group correct-wrong contrast into process supervision by distilling a teacher distribution from the shortest correct completion into prefixes of the longest wrong completion, improving GRPO on AIM...
-
LLMs Know When They Know, but Do Not Act on It: A Metacognitive Harness for Test-time Scaling
A metacognitive harness uses LLMs' pre- and post-solution self-monitoring signals to control test-time reasoning, raising pooled accuracy from 48.3% to 56.9% on text, code, and multimodal benchmarks.
-
REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations
REALISTA generates semantically coherent adversarial prompts via latent-space optimization over input-dependent editing directions, achieving stronger hallucination elicitation than prior realistic attacks on open-sou...
-
Multi-Rollout On-Policy Distillation via Peer Successes and Failures
MOPD improves on-policy distillation by using peer successes and failures from multiple rollouts to construct more informative teacher signals, yielding consistent gains over baselines on reasoning benchmarks.
-
Taming Extreme Tokens: Covariance-Aware GRPO with Gaussian-Kernel Advantage Reweighting
Covariance-weighted GRPO with Gaussian-kernel reweighting tames extreme tokens to stabilize training and boost reasoning performance over standard GRPO.
-
Evaluating the False Trust Engendered by LLM Explanations
A user study finds that LLM reasoning traces and post-hoc explanations create false trust by increasing acceptance of incorrect answers, whereas contrastive dual explanations improve users' ability to detect errors.
-
AIPO: Learning to Reason from Active Interaction
AIPO trains LLMs to expand their reasoning capability boundary via active multi-agent interaction with Verify, Knowledge, and Reasoning agents during RLVR, using importance sampling and clipping to handle feedback, th...
-
AIPO: Learning to Reason from Active Interaction
AIPO adds active multi-agent consultation (Verify, Knowledge, Reasoning agents) plus custom importance sampling to RLVR training so LLMs expand their reasoning boundary and then operate without the agents.
-
Revisiting Transformer Layer Parameterization Through Causal Energy Minimization
CEM recasts Transformer layers as energy minimization steps, enabling constrained parameterizations like weight sharing and low-rank interactions that match standard baselines in 100M-scale language modeling.
-
Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key
RL training compute for logical reasoning follows a power law with horizon depth whose exponent rises with logical expressiveness, yielding better downstream transfer when models train on richer logics.
-
Gradient Extrapolation-Based Policy Optimization
GXPO approximates longer local lookahead in GRPO training via gradient extrapolation from two optimizer steps using three backward passes total, improving pass@1 accuracy by 1.65-5.00 points over GRPO and delivering u...
-
VIDA: A dataset for Visually Dependent Ambiguity in Multimodal Machine Translation
VIDA provides 2,500 visually-dependent ambiguous MT instances and LLM-judge metrics; chain-of-thought SFT improves disambiguation accuracy over standard SFT, especially out-of-distribution.
-
VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Model
VLA-ATTC equips VLA models with adaptive test-time compute via an uncertainty clutch and relative action critic, cutting failure rates by over 50% on LIBERO-LONG.
-
How Fast Should a Model Commit to Supervision? Training Reasoning Models on the Tsallis Loss Continuum
Tsallis q-loss continuum enables faster cold-start escape in reasoning model training via probability-based gradient amplification, with practical Monte Carlo estimators GARL and PAFT.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.