Cast a Wider Net: Coordinated Pass@K Policy Optimization for Code Reasoning

Suman Banerjee; Tong Che; Yilong Li

REVIEW 2 major objections 1 minor 10 references

Coordinated planning of four distinct high-level methods improves pass@4 in code generation over independent sampling.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-29 18:35 UTC pith:GQY37BAE

load-bearing objection CPPO's planner for distinct strategy tuples is a clean way to target redundancy in pass@K, but the gains rest on an untested assumption that the tuples actually differ and the abstract gives too little experimental detail to judge. the 2 major comments →

arxiv 2605.27000 v2 pith:GQY37BAE submitted 2026-05-26 cs.CL cs.AI

Cast a Wider Net: Coordinated Pass@K Policy Optimization for Code Reasoning

Yilong Li , Suman Banerjee , Tong Che This is my paper

classification cs.CL cs.AI

keywords code generationpass@Kpolicy optimizationplanningreinforcement learningtest-time computestrategy explorationverifier

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Standard pass@K sampling draws K independent answers from one distribution, so many attempts collapse onto near-duplicate reasoning paths and waste the fixed solver budget. This paper introduces Coordinated Pass@K Policy Optimization (CPPO), in which a planner first emits a tuple of K=4 alternative high-level methods and a shared solver then produces one solution per method. The joint policy is trained with a multiplicative planner reward that assigns credit only to tuples that yield at least one verifier-confirmed correct answer. Across APPS, CodeContests, and LiveCodeBench-v6, the method raises pass@4 over direct sampling, planning baselines, planner-only SFT, and prior pass@K RL under the same K=4 budget, with statistically significant gains on six of nine model-benchmark cells and a largest observed lift of 0.16.

Core claim

CPPO turns pass@K generation into joint exploration over strategies: a planner emits a tuple of K=4 alternative high-level methods, a shared solver attempts one solution per method, and the joint policy receives a multiplicative planner reward R_plan = J_ψ · R_out that credits only valid strategy tuples leading to verifier-confirmed pass@K success. The resulting policy improves pass@4 over direct sampling, planning baselines, planner-only SFT, and pass@K-oriented RL under identical K=4 solver-attempt budgets, with statistically significant gains on six of nine model-benchmark cells.

What carries the argument

The coordinated planner that outputs a tuple of K alternative high-level methods, trained with multiplicative planner reward to encourage coverage of distinct solution strategies.

Load-bearing premise

The planner can emit a tuple of K=4 alternative high-level methods that are sufficiently distinct to cover different solution strategies.

What would settle it

Run the planner on held-out problems and count how often the K emitted methods are semantically or structurally distinct; if diversity stays low yet pass@4 still rises, the coordination mechanism would be falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

CPPO's planner for distinct strategy tuples is a clean way to target redundancy in pass@K, but the gains rest on an untested assumption that the tuples actually differ and the abstract gives too little experimental detail to judge.

read the letter

The paper's core move is to replace independent sampling with a planner that outputs a tuple of four high-level methods, then runs one solver attempt per method and credits the planner only when the whole tuple passes verification. The multiplicative reward R_plan = J_ψ · R_out is a straightforward way to make the planner care about coverage rather than single successes. That framing is new relative to the pass@K RL baselines they cite, and it directly attacks the known issue that repeated draws from one model often repeat the same algorithmic path on problems that have several viable approaches.

What works is the problem statement and the policy class itself. Turning pass@K into explicit joint exploration over strategies is a reasonable response to the budget waste they describe, and the empirical claim is at least directionally consistent across the three benchmarks.

The soft spots are the ones the stress-test note flags. Nothing in the abstract shows that the emitted tuples are meaningfully distinct rather than near-duplicates; there are no diversity metrics, no examples of the methods, and no ablation that removes any implicit diversity pressure. If the planner collapses, the method reduces to ordinary sampling under the same K=4 budget and the reported edges over direct sampling and PKPO disappear. The numerical results also come with almost no supporting information on run counts, baseline re-implementations, or data handling, so the paired bootstrap p-values are hard to interpret. Those gaps are material for a claim that rests on a 0.16 absolute lift.

This is for labs already running test-time scaling or RL on code models who want to try coordinating attempts. A reader could extract the planner-solver split and the reward form and test it themselves. The work shows clear thinking about the failure mode of independent sampling, so it deserves a serious referee who can check the full experiments and the actual strategy tuples. I would send it to review rather than desk reject.

Referee Report

2 major / 1 minor

Summary. The paper proposes Coordinated Pass@K Policy Optimization (CPPO) for code generation. A planner emits a tuple of K=4 alternative high-level methods; a shared solver produces one implementation per method. The joint policy is trained via the multiplicative reward R_plan = J_ψ · R_out, which credits only tuples that yield verifier-confirmed pass@K success. The central claim is that this yields higher pass@4 than direct sampling, planning baselines, planner-only SFT, and PKPO under identical K=4 budgets, with statistically significant gains (paired bootstrap p<0.05) on six of nine model-benchmark cells and a largest gain of +0.16 (0.588→0.748) on Qwen3.5-9B LiveCodeBench-v6.

Significance. If the gains are attributable to coordinated exploration of distinct algorithmic strategies rather than other factors, the method offers a concrete way to reduce redundant rollouts when allocating fixed test-time compute for code reasoning, which is relevant for competitive-programming settings where multiple solution strategies exist.

major comments (2)

[Abstract and §3] Abstract and §3 (method): The reported pass@4 gains over PKPO and direct sampling rest on the assumption that the planner emits K=4 sufficiently distinct high-level methods. The reward R_plan = J_ψ · R_out contains no explicit diversity term or regularization, and the manuscript provides no diversity metrics, strategy-similarity ablations, or qualitative analysis of emitted tuples. Without such evidence it is unclear whether the joint policy differs meaningfully from independent sampling under the same budget.
[Experiments] Experiments section (presumably §4–5): The abstract states specific numerical gains and paired-bootstrap p<0.05, yet supplies no information on the number of independent runs, exact baseline implementations (including PKPO), or train/test splits. These details are required to evaluate the reliability of the statistical claims and the reproducibility of the largest reported improvement (+0.16).

minor comments (1)

[Abstract] Notation for J_ψ and the precise definition of the planner output tuple could be introduced earlier to improve readability for readers outside RL-for-code.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. Below we respond point-by-point to the major comments, clarifying the design rationale where possible and indicating revisions that will be made to address gaps in evidence and reproducibility.

read point-by-point responses

Referee: [Abstract and §3] Abstract and §3 (method): The reported pass@4 gains over PKPO and direct sampling rest on the assumption that the planner emits K=4 sufficiently distinct high-level methods. The reward R_plan = J_ψ · R_out contains no explicit diversity term or regularization, and the manuscript provides no diversity metrics, strategy-similarity ablations, or qualitative analysis of emitted tuples. Without such evidence it is unclear whether the joint policy differs meaningfully from independent sampling under the same budget.

Authors: The joint policy is structured so that the planner must emit a tuple of K=4 methods whose collective coverage yields verifier success under the multiplicative reward; this is distinct from independent sampling from a single distribution. Nevertheless, we agree that the absence of diversity metrics, similarity ablations, or qualitative tuple examples leaves the degree of coordination unverified. We will add these analyses (pairwise strategy similarity, entropy over method types, and example tuples) to §3 and the appendix in the revision. revision: yes
Referee: [Experiments] Experiments section (presumably §4–5): The abstract states specific numerical gains and paired-bootstrap p<0.05, yet supplies no information on the number of independent runs, exact baseline implementations (including PKPO), or train/test splits. These details are required to evaluate the reliability of the statistical claims and the reproducibility of the largest reported improvement (+0.16).

Authors: We acknowledge that these implementation and statistical details were omitted. In the revised manuscript we will expand the experimental section with the number of independent runs performed, the precise re-implementation of PKPO and other baselines under the K=4 budget, the exact train/test splits for each benchmark, and the bootstrap procedure used for the reported p-values. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical RL procedure with external benchmarks

full rationale

The paper defines CPPO as a joint policy optimization method using a multiplicative reward R_plan = J_ψ · R_out and reports pass@4 gains from direct experiments on APPS, CodeContests, and LiveCodeBench-v6 against multiple baselines under matched K=4 budgets. No equations, predictions, or uniqueness claims reduce the reported improvements to quantities defined by the method's own fitted parameters or self-citations; the evaluation relies on verifier-confirmed success rates measured independently of the training objective. The load-bearing assumption about strategy diversity is an empirical hypothesis tested via outcomes rather than a definitional tautology.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 1 invented entities

Based on abstract only; the method rests on a new policy structure and reward definition whose details are not fully specified.

free parameters (1)

K
Number of coordinated attempts fixed at 4 for all reported experiments.

axioms (1)

domain assumption Distinct high-level methods produced by the planner lead to non-redundant solution attempts
The core motivation and reward design assume that planner tuples avoid the collapse onto near-duplicate paths described in the problem statement.

invented entities (1)

Coordinated planner-solver joint policy no independent evidence
purpose: To perform joint exploration over K alternative strategies instead of independent sampling
New policy class introduced to address the redundancy failure mode.

pith-pipeline@v0.9.1-grok · 5822 in / 1458 out tokens · 38184 ms · 2026-06-29T18:35:47.644678+00:00 · methodology

0 comments

read the original abstract

Repeated sampling with a verifier is the standard way to allocate test-time compute for code generation, with pass@$K$ as the canonical metric. Yet the standard policy class draws $K$ independent samples from a single answer distribution, so attempts often collapse onto near-duplicate reasoning paths and waste the budget on redundant rollouts. This failure is costly in competitive programming, where many problems admit multiple distinct algorithmic strategies and pass@$K$ requires only one correct attempt. We propose Coordinated Pass@$K$ Policy Optimization (CPPO), which turns pass@$K$ generation into joint exploration over strategies: a planner emits a tuple of $K{=}4$ alternative high-level methods, and a shared solver attempts one solution per method. CPPO trains this joint policy with a multiplicative planner reward, $R_{\mathrm{plan}} = J_\psi \cdot R_{\mathrm{out}}$, assigning credit only to valid strategy tuples that lead to verifier-confirmed pass@$K$ success. Across APPS, CodeContests, and LiveCodeBench-v6, CPPO improves pass@$4$ over direct sampling, planning baselines, planner-only SFT, and pass@$K$-oriented RL under the same $K{=}4$ solver-attempt budget, with statistically significant gains on six of nine model--benchmark cells. The largest single gain is $+0.16$ on Qwen3.5-9B LiveCodeBench-v6 over the strongest baseline, PKPO ($0.588 \rightarrow 0.748$; paired bootstrap, $p < 0.05$).

Figures

Figures reproduced from arXiv: 2605.27000 by Suman Banerjee, Tong Che, Yilong Li.

**Figure 1.** Figure 1: Overview of Coordinated Pass@K Policy Optimization. The planner qΘ emits a strategy tuple S = (s1, . . . , sK); the shared solver pΘ produces one solution per strategy; a verifier returns per-branch outcomes ri ∈ {0, 1}, and the outcome reward Rout = maxi ri scores pass@K success. A frozen reward model Jψ(x, S) gates plan validity, giving the planner reward Rplan = Jψ(x, S) · Rout, which is nonzero only wh… view at source ↗

**Figure 2.** Figure 2: LiveCodeBench-v6 pass@K for Qwen3.5-9B (4B counterpart in [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Figure 3: Reward-model diagnostics. (a) The RM matches held-out judge labels but weakly predicts frozen-solver outcomes, supporting its use as a validity gate rather than a plan-quality scorer. (b) Joint distribution of validity decisions and solver outcomes during CPPO rollouts; shading encodes rollout frequency, and only the accepted, solved cell yields nonzero Rplan [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

**Figure 4.** Figure 4: Relationship between algorithmic diversity [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗

**Figure 5.** Figure 5: LiveCodeBench-v6 pass@K across baselines for Qwen3.5-4B and Qwen3.5-9B. of comparable size (Direct Solve 0.640/0.750 on E2B/E4B versus 0.420/0.515 on Qwen3.5-2B/4B), so the absolute CPPO margin over Direct Solve compresses (+0.108/ + 0.082 on E2B/E4B, versus +0.126/ + 0.279 on Qwen3.5-2B/4B) – the dominant headroom on APPS is in the strongest baseline rather than the base model. The LCBv6 pattern matches … view at source ↗

**Figure 6.** Figure 6: APPS pass@K across Qwen3.5-2B, Qwen3.5-4B, and Qwen3.5-9B. Each method occupies a single color, with darker shades for larger models. K = 1 K = 2 K = 4 K = 8 K = 16 K 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 p a s s @ K (A P P S) Direct Solve-4B Direct Solve-9B Plan-and-Solve-4B Plan-and-Solve-9B PlanSearch-4B PlanSearch-9B PKPO-4B PKPO-9B CPPO-4B CPPO-9B [PITH_FULL_IMAGE:figures/full_fig_p022_6.png] view at source ↗

**Figure 7.** Figure 7: Grouped APPS pass@K for Qwen3.5-4B and Qwen3.5-9B at K ∈ {1, 2, 4, 8, 16}. Each method occupies a single color across both sizes, with a lighter shade for 4B and a darker shade for 9B. ing proceeds, we periodically refresh the rewardmodel dataset: we label outputs from the updated planner with the same judge, append them to the training pool, rebalance by prompt and pass/fail label, and resume finetuning… view at source ↗

**Figure 8.** Figure 8: PKPO training dynamics for Qwen3.5-2B, 4B, and 9B over 30 epochs (light to dark red): gradient [PITH_FULL_IMAGE:figures/full_fig_p025_8.png] view at source ↗

**Figure 9.** Figure 9: UpSkill training dynamics for Qwen3.5-2B, 4B, and 9B over 30 epochs (light to dark purple): gradient L2 [PITH_FULL_IMAGE:figures/full_fig_p025_9.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

10 extracted references · 2 canonical work pages · 1 internal anchor

[1]

Training Verifiers to Solve Math Word Problems

Training verifiers to solve math word prob- lems.arXiv preprint arXiv:2110.14168. Google DeepMind. 2026. Gemma 4: Open lightweight models. Official model cards, https://ai. google.dev/gemma. Gemma 4 E2B and E4B. LiveCodeBench-v6 pass@4 values used in this paper (44.0% for E2B, 52.0% for E4B) reproduced from the official model-card evaluation tables. Acces...

work page internal anchor Pith review Pith/arXiv arXiv 2026
[2]

arXiv preprint arXiv:2509.02534 , year=

DeepSeek-R1: Incentivizing reasoning capa- bility in LLMs via reinforcement learning.Nature, 645:633–638. Dan Hendrycks, Steven Basart, Saurav Kadavath, Man- tas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. 2021. Measuring coding challenge com- petence with APPS. InAdvances in Neural Infor- matio...

work page arXiv 2021
[3]

The same base-model solver is given a prompt that asks it to first write a solution plan and then write the code
[4]

For each problem we sample K full solution attempts under that prompt
[5]

Each attempt passes through the same code extraction, sandboxed execution, and verifier pipeline
[6]

This comparison isolates the effect of adding a planning prompt before code generation

Pass@K is the fraction of problems on which at least one of theKattempts passes the tests. This comparison isolates the effect of adding a planning prompt before code generation. PlanSearch (Wang et al., 2024) as a planning baseline.We follow the released implementation, using eight candidate plans per problem and the same solver/verifier pipeline as the ...

2024
[7]

The base model first generates multiple candi- date plans per problem; we use8candidates
[8]

The candidates are selected and organized into plans usable for solving
[9]

The same frozen base-model solver then gener- ates code conditioned on each selected plan
[10]

This comparison isolates inference-time multi-plan search without planner training

Each problem is evaluated under a budget of K solver attempts, with the same verifier and pass@Kdefinition as above. This comparison isolates inference-time multi-plan search without planner training. PKPO (Walder and Karkhanis, 2025) as a pass@K RL baseline.PKPO transforms the per- sample reward vector r∈ {0,1} n over n≥k samples of the same problem into...

2025

[1] [1]

Training Verifiers to Solve Math Word Problems

Training verifiers to solve math word prob- lems.arXiv preprint arXiv:2110.14168. Google DeepMind. 2026. Gemma 4: Open lightweight models. Official model cards, https://ai. google.dev/gemma. Gemma 4 E2B and E4B. LiveCodeBench-v6 pass@4 values used in this paper (44.0% for E2B, 52.0% for E4B) reproduced from the official model-card evaluation tables. Acces...

work page internal anchor Pith review Pith/arXiv arXiv 2026

[2] [2]

arXiv preprint arXiv:2509.02534 , year=

DeepSeek-R1: Incentivizing reasoning capa- bility in LLMs via reinforcement learning.Nature, 645:633–638. Dan Hendrycks, Steven Basart, Saurav Kadavath, Man- tas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. 2021. Measuring coding challenge com- petence with APPS. InAdvances in Neural Infor- matio...

work page arXiv 2021

[3] [3]

The same base-model solver is given a prompt that asks it to first write a solution plan and then write the code

[4] [4]

For each problem we sample K full solution attempts under that prompt

[5] [5]

Each attempt passes through the same code extraction, sandboxed execution, and verifier pipeline

[6] [6]

This comparison isolates the effect of adding a planning prompt before code generation

Pass@K is the fraction of problems on which at least one of theKattempts passes the tests. This comparison isolates the effect of adding a planning prompt before code generation. PlanSearch (Wang et al., 2024) as a planning baseline.We follow the released implementation, using eight candidate plans per problem and the same solver/verifier pipeline as the ...

2024

[7] [7]

The base model first generates multiple candi- date plans per problem; we use8candidates

[8] [8]

The candidates are selected and organized into plans usable for solving

[9] [9]

The same frozen base-model solver then gener- ates code conditioned on each selected plan

[10] [10]

This comparison isolates inference-time multi-plan search without planner training

Each problem is evaluated under a budget of K solver attempts, with the same verifier and pass@Kdefinition as above. This comparison isolates inference-time multi-plan search without planner training. PKPO (Walder and Karkhanis, 2025) as a pass@K RL baseline.PKPO transforms the per- sample reward vector r∈ {0,1} n over n≥k samples of the same problem into...

2025