Pith. sign in

REVIEW 5 major objections 8 minor 60 cited by

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · claude-opus-4-7

A language model that writes its reasoning into the same stream as its actions plans, retrieves, and recovers from mistakes better than one that does either alone.

2026-05-09 01:01 UTC pith:VTZ5TICI

load-bearing objection Solid, influential paper introducing the thought/action/observation loop; the within-LLM ablations carry the claim, the "+34/+10 vs IL/RL" framing oversells it. the 5 major comments →

arxiv 2210.03629 v3 pith:VTZ5TICI submitted 2022-10-06 cs.CL cs.AIcs.LG

ReAct: Synergizing Reasoning and Acting in Language Models

Shunyu Yao , Jeffrey Zhao , Dian Yu , Nan Du , Izhak Shafran , Karthik Narasimhan , Yuan Cao This is my paper
classification cs.CL cs.AIcs.LG
keywords large language modelschain-of-thought reasoningtool useinteractive decision makingin-context learningagent promptingretrieval-augmented reasoningembodied language agents
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks what happens if a language-model agent is allowed to interleave private reasoning with environment actions in a single generated trajectory, instead of either reasoning in isolation (chain-of-thought) or acting without verbalised thought. The authors show that a frozen large model, prompted with a few hand-written examples that demonstrate this interleaving, will spontaneously decompose tasks, track subgoals, retrieve information when its internal knowledge is thin, and revise plans when an observation contradicts an earlier thought. On a multi-hop question-answering task with a small Wikipedia API, this matches or modestly beats chain-of-thought, and a hybrid that backs off between the two does best. On a household text game and a shopping environment, the same pattern with one or two demonstrations beats agents trained on orders of magnitude more data. The contribution is a prompting pattern, an interpretation of why it works (reasoning steers retrieval; retrieval grounds reasoning), and evidence that the resulting trajectories are auditable and editable by a human.

Core claim

The paper argues that reasoning and acting should not be separate prompting modes for a language model. By letting a frozen LLM emit free-form "thoughts" in the same token stream as environment actions, the model can plan, revise plans, decide what to look up next, and incorporate what it just observed, all without any fine-tuning. The authors claim this simple change closes the loop between internal deliberation and external feedback: thoughts decide what to do, observations correct the thoughts, and the next thought adjusts the plan. They report this outperforms reasoning-only and acting-only prompting on a question-answering benchmark with a Wikipedia API, and on two interactive benchmark

What carries the argument

An augmented action space in which the agent's policy emits, at each step, either an external action (search, click, navigate, manipulate) or a "thought" — a free-form natural-language token sequence that produces no observation but updates the context the next decision is conditioned on. The same frozen language model generates both kinds of tokens, prompted with a handful of human-written trajectories that show when to think and when to act. Reasoning is treated as an internal action rather than a separate phase.

Load-bearing premise

That the improvements come from the interleaving pattern itself rather than from hand-tuned prompts, best-of-several prompt selection, and the very large base model — the head-to-head against trained agents mixes a method change with a scale change.

What would settle it

Re-run the four-way ablation (Standard, CoT, Act, ReAct) under matched, randomly drawn prompt sets across multiple seeds and across base models of varying scale, on held-out task splits not used for prompt selection. If ReAct's advantage on ALFWorld and WebShop disappears once "best-of-six" prompt selection is removed, or if a smaller base model with ReAct fails to beat a similarly sized imitation-learned agent, then the gains are attributable to scale and prompt curation rather than to interleaving reasoning with action.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A general-purpose agent loop can be built from prompting alone if the action vocabulary is extended to include unconstrained natural-language thoughts that do not touch the environment.
  • Grounding reasoning in retrieved observations reduces fabrication: on fact verification, the interleaved variant produces fewer hallucinated supporting claims than chain-of-thought reasoning that runs without external lookup.
  • Combining internal reasoning with external retrieval works best as a fallback ensemble: use reasoning-with-self-consistency when the model is confident, fall back to the acting variant when it is not, and vice versa.
  • Human oversight becomes lightweight: editing one or two thoughts mid-trajectory can redirect the agent's whole plan, because the thoughts are the policy's exposed control surface.
  • Fine-tuning a small model on a few thousand interleaved thought-action traces can outperform a much larger model prompted in any of the non-interleaved styles, suggesting the pattern is learnable, not just elicitable.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The thought channel functions as an exposed working memory; this suggests that the right primitive for agent design is not 'plan then execute' but a single token stream where deliberation and action are interchangeable, which has implications for how future agent training data should be collected.
  • The reported gap on interactive benchmarks compares a 540B-parameter prompted model against far smaller imitation-trained agents, so part of what is being measured is the value of pretrained world knowledge, not only the value of interleaving — a controlled comparison at matched scale would clarify how much of the effect is the pattern itself.
  • Because thoughts are human-readable and editable mid-trajectory, this style of agent admits a form of supervision that gradient-based agents cannot easily support: correcting a single sentence redirects the policy, which may matter more for deployment than the headline benchmark numbers.
  • The failure mode where the model loops on repeated thoughts hints that greedy decoding plus an unbounded thought channel can trap the agent; sampling, beam search over thoughts, or an explicit 'give up and retry' action are natural next steps.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. The paper proposes ReAct, a prompting paradigm in which a frozen LLM (PaLM-540B, with GPT-3 confirmation in Appendix A.1) interleaves free-form natural-language "thoughts" with domain-specific actions and observations within a single trajectory. The action space is augmented to = A ∪ L, where thoughts are no-op tokens that update context but not the environment. ReAct is evaluated on four benchmarks: HotpotQA and FEVER (with a simple Wikipedia search/lookup/finish API), ALFWorld, and WebShop. The authors report (i) on knowledge tasks, ReAct underperforms CoT on HotpotQA EM (27.4 vs 29.4) but outperforms it on FEVER (60.9 vs 56.3), and a ReAct↔CoT-SC fallback combination beats both; (ii) on ALFWorld, ReAct (best-of-6 prompt permutations) reaches 71% success vs 45% for Act and 37% for BUTLER; (iii) on WebShop, ReAct reaches 40.0% success vs 30.1% for Act and ≤29.1% for IL/IL+RL baselines; (iv) finetuning small PaLM models on 3,000 ReAct trajectories outperforms finetuning on Standard/CoT/Act trajectories. Ablations against an IM-style "inner monologue" baseline (ReAct-IM) and a human error analysis on HotpotQA are also reported.

Significance. If the central claim holds — that interleaving reasoning and acting in one decoded trajectory yields a robust prompting pattern across both knowledge-intensive QA and embodied/web decision-making — this is a useful and general contribution. The evidence is strongest where it matters most: the within-model ablations on a frozen PaLM-540B (ReAct vs Act on ALFWorld, +26 absolute best-of-6; ReAct vs Act on WebShop, +9.9 SR; ReAct vs ReAct-IM, +18 on ALFWorld) isolate the contribution of the prompting pattern from model scale. The qualitative human error analysis (Table 2) is informative and honest about ReAct's failure modes (47% reasoning errors, 23% search errors). The combined ReAct↔CoT-SC strategy and the finetuning scaling results (Fig. 3), where small ReAct-finetuned models beat much larger CoT/Standard-prompted models, give the paper additional evidentiary depth beyond a single prompt comparison. The paradigm has since been broadly adopted, and the explicit thought-edit human-in-the-loop demonstration (Fig. 5) is a genuine interpretability dividend. Prompts and code are released, supporting reproducibility within the constraints of a closed model.

major comments (5)
  1. [Abstract / §4, Tables 3–4] The headline framing 'outperforms imitation and RL methods by an absolute success rate of 34% and 10%' conflates the contribution of the ReAct prompting pattern with the contribution of using a 540B LLM as the policy. Decomposing Table 3, BUTLER=37, Act(best-of-6)=45, ReAct(best-of-6)=71: roughly 8 of the 34 absolute points on ALFWorld come from 'replace small IL agent with PaLM-540B + actions only', and only ~26 from the ReAct pattern itself. On WebShop the decomposition is more favorable (IL=29.1, Act=30.1, ReAct=40.0), so +10 is mostly method. The within-model ablations are reported and support the method, but the abstract should report the within-LLM gap (Act→ReAct) alongside the cross-method gap so readers do not mis-attribute scale effects.
  2. [§4, Table 3 (ALFWorld)] ALFWorld headline numbers are best-of-6 prompt permutations for ReAct/Act/ReAct-IM, while BUTLER is best-of-8 with beam search and ReAct uses greedy decoding. The mixture of selection criteria (best-of-K over prompt permutations vs best-of-K over beam search) makes the head-to-head comparison less clean than presented. The authors do also report ReAct(avg)=57 vs Act(avg presumably similar to or below best-of-6=45) and ReAct-IM(avg)=48, which is the most convincing comparison and should be emphasized; please also report Act(avg) explicitly and the variance/std across the 6 prompts so readers can judge prompt-selection sensitivity.
  3. [§3, Table 1 / Fig. 2] Variance is not reported for HotpotQA/FEVER prompting results. Given that the EM gap between ReAct (27.4) and CoT (29.4) is 2.0 points and the gap between ReAct→CoT-SC (35.1) and CoT-SC (33.4) is 1.7 points, single-run numbers without seed variance or confidence intervals make it hard to judge whether the ordering is robust. Please report at least decoding-seed variance for the non-greedy methods and, ideally, a bootstrap CI on the 500-question (or full) eval subset.
  4. [§3.2 'Combining Internal and External Knowledge'] The CoT-SC↔ReAct fallback heuristic introduces hyperparameters (max steps = 7/5; majority threshold n/2 over 21 samples) that are tuned on the same evaluation distribution. The text says 'we find more steps will not improve ReAct performance' but does not state whether this was determined on a held-out split. Please clarify whether these thresholds were selected on dev or test, and report sensitivity (e.g., performance vs. step budget and vs. threshold) so the combined-method gains in Table 1 are not effectively dev-tuned on test.
  5. [§3.3 Table 2 (error analysis)] The human-labeled success/failure analysis is one of the strongest pieces of evidence in the paper but the methodology is under-specified. Please state: how many annotators, inter-annotator agreement, whether annotators were blind to which method produced each trajectory, and the exact sampling protocol (the text says '50 correct and 50 incorrect from each method' which oversamples failures relative to base rates). The qualitative claim that CoT hallucinates 56% of the time vs ReAct 0% is a strong one and deserves a more rigorous protocol description.
minor comments (8)
  1. [§2] The augmented action space = A ∪ L is introduced as a conceptual device, but L is unbounded and the policy π is left abstract. A sentence clarifying that in practice π is the LLM's next-token distribution conditioned on the full trajectory, and that 'thought vs action' is determined by a parsing convention on the decoded text, would help readers reproduce the system.
  2. [Figure 1] The figure uses encoded/garbled glyphs in the rendered PDF (visible as e.g. '$FW\u0003\u0014\u001d\u00037KLQN'); please re-embed the fonts so the example trajectories are legible. This is the paper's signature figure.
  3. [§3.1 Action Space] The Wikipedia API returns 'the first 5 sentences' of a page. This is an arbitrary cutoff that interacts with the search-error rate (23% of ReAct failures). Please state whether this was tuned, and whether longer returns hurt or help.
  4. [§4 WebShop] Score and SR are reported as point values without confidence intervals on 500 test instructions. A binomial CI would clarify whether the IL+RL→ReAct gap of ~11 SR points is comfortably outside noise.
  5. [Appendix B.1] Finetuning step counts (4000 for ReAct/Act, 1000–2000 for Standard/CoT 'because the latter degrade soon after') deserve a learning-curve plot rather than a one-line justification, since the comparison in Fig. 3 depends on this choice.
  6. [§5] Related work could more explicitly contrast with Inner Monologue beyond Section 4's qualitative remark — e.g., a single sentence noting that IM operates on real robotic affordances while ReAct operates on text environments would prevent over-claiming.
  7. [Throughout] 'Reasoning traces' and 'thoughts' are used interchangeably; pick one term and use it consistently in formal statements (Section 2) and tables.
  8. [Appendix A.2 / Fig. 4] The 'outdated label' anecdote on HotpotQA is interesting but only one example is shown. Either expand to a small audit (how many of the 50 'correct' ReAct samples disagree with gold for label-staleness reasons?) or label this clearly as anecdotal.

Simulated Author's Rebuttal

5 responses · 1 unresolved

We thank the referee for the careful and constructive report, and for the recommendation to accept. The major comments target presentation and methodological-rigor issues rather than the core claims, and we agree with all five. In the revision we will: (1) decompose the headline gains in the abstract into within-LLM (Act→ReAct) and cross-method components so scale effects and prompting-pattern effects are not conflated; (2) add Act(avg) and per-prompt standard deviations to Table 3, lead with the avg-vs-avg comparison, and footnote the BUTLER selection-protocol difference; (3) add bootstrap CIs and seed variance to Table 1 and Fig. 2, and be explicit about which orderings are within overlap; (4) clarify that step-budget and majority-threshold hyperparameters for the ReAct↔CoT-SC combination were set on dev, and add a sensitivity sweep; (5) document the Table 2 error-analysis protocol (author-labeled, non-blinded, stratified 50+50 sampling, percentages conditional within stratum) and run a blinded multi-annotator re-labeling with reported agreement for the camera-ready. We list as a standing objection that we cannot guarantee the exact 56%/0% hallucination figures will replicate under the stricter blinded protocol, though we expect the qualitative direction to hold.

read point-by-point responses

  1. Referee: Headline framing conflates ReAct prompting contribution with the contribution of using PaLM-540B; on ALFWorld, ~8 of 34 absolute points come from 'replace small IL agent with PaLM-540B + actions only'. The abstract should report the within-LLM Act→ReAct gap alongside the cross-method gap.

    Authors: We agree that the abstract framing should not let readers conflate scale effects with the prompting pattern. We will revise the abstract and §4 introduction to report the within-LLM Act→ReAct gap alongside the cross-method gap. Concretely, the revised abstract sentence will read approximately: 'On ALFWorld and WebShop, ReAct outperforms a same-LLM Act-only baseline by 26 and 10 absolute points in success rate respectively, and outperforms imitation/RL baselines by 34 and 10 points.' We will mirror this decomposition in the §4 results paragraph and in the captions of Tables 3 and 4. The within-model ablations are already the headline of our analysis (cf. §4 'On the value of internal reasoning vs. external feedback'), and we are happy to surface them more prominently in the abstract. The referee's decomposition is consistent with our own reading of the data. revision: yes

  2. Referee: ALFWorld numbers mix selection criteria (best-of-6 prompt permutations for ReAct/Act/ReAct-IM, best-of-8 beam search for BUTLER, greedy decoding for ReAct). Please report Act(avg) explicitly and per-prompt variance/std, and emphasize the avg-vs-avg comparison.

    Authors: The point is well taken. The most controlled comparison in our setup is indeed ReAct(avg) vs Act(avg) vs ReAct-IM(avg) under matched greedy decoding and matched prompt-permutation protocol; BUTLER's best-of-8 beam search is reported as taken from Shridhar et al. (2020b) and is not directly comparable in selection protocol. In the revision we will (i) add Act(avg) across the 6 prompt permutations to Table 3, (ii) add std across permutations for ReAct, Act, and ReAct-IM in each task category and overall, and (iii) rewrite the §4 narrative to lead with the avg-vs-avg comparison and present best-of-6 as a secondary, prompt-selection-sensitivity figure. We will also add a footnote clarifying the BUTLER selection protocol difference so the head-to-head is not over-claimed. Note: ReAct(avg)=57 and ReAct-IM(avg)=48 are already reported; the missing Act(avg) is a presentation gap we will close. revision: yes

  3. Referee: Variance is not reported for HotpotQA/FEVER prompting results; with EM gaps of ~2 points (ReAct 27.4 vs CoT 29.4; ReAct→CoT-SC 35.1 vs CoT-SC 33.4), single-run numbers make it hard to judge robustness. Please report decoding-seed variance and ideally bootstrap CIs.

    Authors: We agree that with gaps of ~2 EM points, point estimates alone are insufficient. In the revision we will add: (i) bootstrap 95% confidence intervals over the evaluation set (resampling questions) for all rows of Table 1, which is the appropriate measure of evaluation-set uncertainty for the deterministic-decoding methods (Standard, CoT, Act, ReAct under greedy); (ii) seed variance across at least 3 sampling seeds for the stochastic CoT-SC and ReAct↔CoT-SC entries, since these draw 21 samples at temperature 0.7. We will also add CI bands to Fig. 2. We will be transparent in the text about which orderings are statistically robust and which (e.g., ReAct vs CoT on HotpotQA EM) are within overlapping intervals; this is consistent with our existing claim that the two methods are complementary rather than that ReAct dominates CoT on HotpotQA. revision: yes

  4. Referee: The CoT-SC↔ReAct fallback heuristic introduces hyperparameters (max steps 7/5; majority threshold n/2) that may have been tuned on the evaluation distribution. Clarify whether thresholds were selected on dev or test, and report sensitivity.

    Authors: Thank you for raising this. To clarify: HotpotQA and FEVER have public dev sets, and our prompting evaluations are on dev (HotpotQA test labels are not public). The step budgets (7 for HotpotQA, 5 for FEVER) were chosen by inspecting the length distribution of correct ReAct trajectories on the training trajectories used to construct prompts (the footnote 'trajectories with 7 steps on HotpotQA and 5 steps on FEVER only take up 0.84% and 1.33%' refers to this). The n/2 majority threshold was not tuned; it is the natural 'no clear majority' cutoff. We acknowledge the text was not explicit on this point. In the revision we will (i) state explicitly which split was used to set the step budget, (ii) add a sensitivity sweep over step budget {3,5,7,10} and over the majority threshold for the combined methods, reported on dev, and (iii) note that all reported numbers correspond to a single fixed (budget, threshold) chosen before the final evaluation runs. If sensitivity is large, we will report the worst-case as well as the chosen-config result. revision: yes

  5. Referee: The Table 2 human error analysis methodology is under-specified: number of annotators, inter-annotator agreement, blinding, and sampling protocol (50 correct + 50 incorrect oversamples failures). The CoT-hallucinates-56%-vs-ReAct-0% claim deserves a rigorous protocol description.

    Authors: The referee is correct that Table 2 is under-documented. To be transparent about what was actually done: the labeling was performed by the authors, with the categories defined collaboratively before labeling and trajectories labeled with method identity visible (i.e., not blinded). Sampling was stratified 50 correct + 50 incorrect per method (200 total) precisely to study failure modes, not to estimate population-level rates; the percentages in Table 2 are conditional within the success or failure stratum, not marginal. We will revise the table caption and §3.3 text to (i) state the labeling protocol and stratified sampling explicitly, (ii) reframe the 56% / 0% numbers as conditional on the failure stratum and as labeler-author estimates rather than population rates, and (iii) note the absence of blinding as a limitation. For the camera-ready/extended version we will additionally run a blinded re-labeling with two independent annotators on a fresh sample and report Cohen's κ; we cannot promise the original 56%/0% figures will exactly replicate, but the qualitative direction (CoT more prone to fact hallucination, ReAct more prone to reasoning loops and uninformative searches) is robust in our reading of the trajectories. revision: partial

standing simulated objections not resolved
  • A fully blinded, multi-annotator re-labeling of the HotpotQA error analysis (Table 2) with reported inter-annotator agreement was not part of the original submission, and we cannot guarantee the exact 56%/0% hallucination figures will replicate under that stricter protocol. We will run the blinded study and report results, but the original numbers should be read as author-labeled, non-blinded estimates conditional on the stratified sample.

Circularity Check

2 steps flagged

No meaningful circularity: ReAct's claims are evaluated on held-out external benchmarks against independent baselines, with within-LLM ablations doing the load-bearing work.

specific steps
  1. self citation load bearing [Section 4 (WebShop), Table 4]
    "We compare to an imitation learning (IL) method trained with 1,012 human annotated trajectories, and a imitation + reinforcement learning (IL + RL) method additionally trained with 10,587 training instructions. ... IL/IL+RL taken from Yao et al. (2022)."

    WebShop environment and the IL/IL+RL baselines are from authors' own prior work (Yao et al. 2022). This is not load-bearing circularity because the metric (success rate, score) is independently defined and the baselines are reported numbers, not values fitted to make ReAct look good. Noted only for completeness; does not raise the score meaningfully.

  2. other [Section 3.2, 'Finetuning' paragraph and Figure 3]
    "we consider a bootstraping approach similar to Zelikman et al. (2022), using 3,000 trajectories with correct answers generated by ReAct (also for other baselines) to finetune smaller language models (PaLM-8/62B) to decode trajectories"

    Finetuning data is filtered by ReAct's own correctness on HotpotQA, so the training distribution is method-shaped. However, the same bootstrapping is applied symmetrically to Standard/CoT/Act baselines, and evaluation EM is against external HotpotQA labels, so within-method comparisons remain fair. Mild self-loop, not load-bearing for the headline claim.

full rationale

The paper's central claim — that interleaving free-form reasoning traces with environment actions improves task performance over reasoning-only or acting-only prompting — is evaluated on four external benchmarks (HotpotQA, FEVER, ALFWorld, WebShop) using metrics (EM, accuracy, success rate) that are defined independently of the method. Baselines include CoT/CoT-SC (Wei et al. 2022; Wang et al. 2022a), Standard prompting, Act-only, BUTLER (Shridhar et al. 2020b), and IL/IL+RL (Yao et al. 2022). None of these are derived from ReAct's own outputs, so there is no self-definitional loop and no fitted-input-called-prediction pattern. Self-citation exists (WebShop is from Yao et al. 2022, an author's prior work), but it provides the environment, not the evaluation criterion or the baseline numbers being beaten — the IL/IL+RL baselines and the score/SR metric come from that prior work and are used as external comparators. This is normal benchmark reuse, not load-bearing self-citation. The reader's skeptical decomposition (that the headline "+34" on ALFWorld conflates ~+26 method gain with ~+8 scale gain over BUTLER, and that ALFWorld headline is best-of-6 prompt permutations) is a valid concern about claim framing and experimental fairness, but it is not circularity. It is a confound/presentation issue: the paper does report ReAct(avg)=57 vs. Act(best-of-6)=45, which is an honest within-LLM comparison and not self-referential. The HotpotQA finetuning experiment (Section 3.2, "bootstrapping approach... using 3,000 trajectories with correct answers generated by ReAct... to finetune smaller language models") trains on ReAct's own correct trajectories and then evaluates ReAct-style decoding. This is a mild self-loop in that the training distribution is method-shaped, but (a) evaluation is still on held-out HotpotQA EM against the same external label set, (b) all four methods (Standard, CoT, Act, ReAct) are finetuned on their own correct trajectories symmetrically, so the comparison is internally fair, and (c) the result is not presented as the headline claim. This warrants noting but not a high score. Overall: derivation chain is empirical, baselines are external, metrics are external. Score 1.

Axiom & Free-Parameter Ledger

3 free parameters · 2 axioms · 1 invented entities

ReAct is an empirical prompting paper, so the 'axiom ledger' is light: it relies on a pretrained frozen LLM (PaLM-540B, also GPT-3) and on the standard assumption that benchmark accuracy on HotpotQA/Fever/ALFWorld/WebShop is a meaningful proxy for reasoning+acting capability. There are no invented physical entities. Free parameters are the manually authored few-shot prompts (3–6 demonstrations per task) and a small number of decoding/length hyperparameters; these are chosen by hand, not fit by gradient, but they are non-trivial degrees of freedom and ALFWorld reports best-of-6 permutations.

free parameters (3)
  • Few-shot demonstrations per task = 6 (HotpotQA), 3 (Fever), 3 per task type × 6 permutations (ALFWorld), 1–2 (WebShop)
    Hand-authored prompts; 'best of 6' permutations reported on ALFWorld means prompt selection is a tunable knob.
  • Max ReAct steps before fallback to CoT-SC = 7 (HotpotQA), 5 (Fever)
    Threshold for the ReAct→CoT-SC hybrid; chosen empirically.
  • CoT-SC sample count and temperature = 21 samples, T=0.7
    Standard self-consistency settings used as baseline and in the hybrid.
axioms (2)
  • domain assumption A frozen pretrained LLM has sufficient latent reasoning ability that few-shot demonstrations of thought+action suffice to elicit useful agent behavior.
    Implicit in the entire prompting paradigm; supported by CoT and SayCan but not proved here.
  • domain assumption Benchmark accuracy on HotpotQA/Fever/ALFWorld/WebShop measures the intended capability ('reasoning + acting').
    Standard in the field; HotpotQA in particular has known label-staleness issues that the authors themselves note (Fig. 4).
invented entities (1)
  • Augmented action space = A ∪ L (language thoughts as no-op actions) independent evidence
    purpose: Formalize interleaving of reasoning tokens and environment actions inside one trajectory.
    Not a physical postulate; it is a definitional move and its usefulness is tested empirically on four benchmarks.

pith-pipeline@v0.9.0 · 9741 in / 5488 out tokens · 92635 ms · 2026-05-09T01:01:59.069339+00:00 · methodology

0 comments
read the original abstract

While large language models (LLMs) have demonstrated impressive capabilities across tasks in language understanding and interactive decision making, their abilities for reasoning (e.g. chain-of-thought prompting) and acting (e.g. action plan generation) have primarily been studied as separate topics. In this paper, we explore the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources, such as knowledge bases or environments, to gather additional information. We apply our approach, named ReAct, to a diverse set of language and decision making tasks and demonstrate its effectiveness over state-of-the-art baselines, as well as improved human interpretability and trustworthiness over methods without reasoning or acting components. Concretely, on question answering (HotpotQA) and fact verification (Fever), ReAct overcomes issues of hallucination and error propagation prevalent in chain-of-thought reasoning by interacting with a simple Wikipedia API, and generates human-like task-solving trajectories that are more interpretable than baselines without reasoning traces. On two interactive decision making benchmarks (ALFWorld and WebShop), ReAct outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples. Project site with code: https://react-lm.github.io

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. LoHoSearch: Benchmarking Long-Horizon Search Agents Beyond the Human Difficulty Ceiling

    cs.CL 2026-06 unverdicted novelty 8.0

    LoHoSearch is a new benchmark of 544 KG-constructed questions across 11 domains where the strongest search agent scores 34.74% and context strategies add at most 6.8%.

  2. What Benchmarks Don't Measure: The Case for Evaluating Abstention Competence in Autonomous Agents

    cs.AI 2026-06 conditional novelty 8.0

    Current benchmarks overlook abstention competence in agents due to compliance bias; a new three-gap taxonomy and metrics (Safety Rate, Usability Rate, Informed Refusal Rate) demonstrate tunable safety-usability tradeo...

  3. SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain

    cs.AI 2026-05 accept novelty 8.0

    SVFSearch is the first open benchmark for short-video frame search in the Chinese gaming domain, with evaluations showing direct QA at 66.4%, best practical agents at 79.1%, and oracle knowledge at 95.4%.

  4. Who Owns This Agent? Tracing AI Agents Back to Their Owners

    cs.CR 2026-05 unverdicted novelty 8.0

    A canary injection protocol for linking observed AI agent behavior to the responsible account at the hosting vendor, with robust variants for adversarial filtering.

  5. Continual Harness: Online Adaptation for Self-Improving Foundation Agents

    cs.LG 2026-05 conditional novelty 8.0

    Continual Harness automates online self-improvement for foundation-model embodied agents by refining prompts, sub-agents, skills, and memory within one run, cutting button-press costs on Pokemon Red and Emerald and cl...

  6. SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning

    cs.AI 2026-05 accept novelty 8.0

    SimWorld Studio deploys an evolving coding agent to create adaptive 3D environments that co-evolve with embodied learners, delivering 18-point success-rate gains over fixed environments in navigation benchmarks.

  7. SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning

    cs.AI 2026-05 unverdicted novelty 8.0

    SimWorld Studio uses a self-evolving coding agent to generate adaptive 3D environments that improve embodied agent performance, with reported gains of 18 points over fixed environments in navigation tasks.

  8. ShadowMerge: A Novel Poisoning Attack on Graph-Based Agent Memory via Relation-Channel Conflicts

    cs.CR 2026-05 unverdicted novelty 8.0

    ShadowMerge poisons graph-based agent memory via relation-channel conflicts using an AIR pipeline, achieving 93.8% average attack success rate on Mem0 and three real-world datasets while bypassing existing defenses.

  9. ShadowMerge: A Novel Poisoning Attack on Graph-Based Agent Memory via Relation-Channel Conflicts

    cs.CR 2026-05 unverdicted novelty 8.0

    ShadowMerge poisons graph-based agent memory by creating relation-channel conflicts that get extracted and retrieved, achieving 93.8% attack success rate on Mem0 and datasets like PubMedQA while evading prior defenses.

  10. ShadowMerge: A Novel Poisoning Attack on Graph-Based Agent Memory via Relation-Channel Conflicts

    cs.CR 2026-05 unverdicted novelty 8.0

    ShadowMerge exploits relation-channel conflicts to poison graph-based agent memory, achieving 93.8% average attack success rate on Mem0 and real-world datasets while bypassing existing defenses.

  11. The Khipu Problem: Institutional Legibility Under Distributed Cognition

    cs.CY 2026-05 unverdicted novelty 8.0

    The khipu problem frames a governance failure in distributed AI where interpretive continuity is lost even when traces remain, requiring infrastructure to preserve reading practices rather than only data retention.

  12. Evaluating Large Language Models in Scientific Discovery

    cs.AI 2025-12 unverdicted novelty 8.0

    The SDE benchmark shows LLMs lag on scientific discovery tasks relative to general science tests, with diminishing scaling returns and shared weaknesses across models.

  13. Agentic AI for Multi-Stage Physics Experiments at a Large-Scale User Facility Particle Accelerator

    physics.acc-ph 2025-09 unverdicted novelty 8.0

    A language-model-driven agentic AI system autonomously executes multi-stage physics experiments at a production synchrotron light source, reducing preparation time by two orders of magnitude while upholding safety con...

  14. ExCyTIn-Bench: Evaluating LLM agents on Cyber Threat Investigation

    cs.CR 2025-07 unverdicted novelty 8.0

    ExCyTIn-Bench is the first benchmark of 7542 questions from Microsoft Sentinel threat investigation graphs, where the best LLM agent achieves a reward of 0.606.

  15. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents

    cs.CR 2024-06 unverdicted novelty 8.0

    AgentDojo introduces an extensible evaluation framework populated with realistic agent tasks and security test cases to measure prompt injection robustness in tool-using LLM agents.

  16. DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines

    cs.CL 2023-10 conditional novelty 8.0

    DSPy compiles short declarative programs into LM pipelines that self-optimize and outperform both standard few-shot prompting and expert-written chains on math, retrieval, and QA tasks.

  17. Mind2Web: Towards a Generalist Agent for the Web

    cs.CL 2023-06 accept novelty 8.0

    Mind2Web is the first large-scale dataset of real-world web tasks for developing generalist language-guided agents that complete complex actions on diverse websites.

  18. Tree of Thoughts: Deliberate Problem Solving with Large Language Models

    cs.CL 2023-05 accept novelty 8.0

    Tree of Thoughts enables language models to solve complex planning tasks by generating, evaluating, and searching over coherent intermediate thoughts in a tree, raising Game of 24 success from 4% to 74% with GPT-4.

  19. API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs

    cs.CL 2023-04 conditional novelty 8.0

    API-Bank is a new benchmark and training dataset for tool-augmented LLMs that shows fine-tuned models can approach GPT-3.5 tool-use effectiveness.

  20. PAL: Program-aided Language Models

    cs.CL 2022-11 conditional novelty 8.0

    PAL improves few-shot reasoning accuracy by having LLMs generate executable programs rather than text-based chains of thought, outperforming much larger models on math and logic benchmarks.

  21. TO-Master: an LLM-agent framework for automated topology optimization

    cs.CE 2026-07 unverdicted novelty 7.0

    TO-Master is an LLM agent framework that orchestrates finite-element topology optimization from conversational inputs, supporting 2D/3D compliance, thermal, stress-constrained, and multi-load cases while reproducing b...

  22. Dynamic Bidirectional Pattern Memory: A Production-Scale Empirical Characterisation of Inference-Time Gating in Clinical NLP

    cs.CL 2026-07 unverdicted novelty 7.0

    Empirical study on production-scale clinical NLP shows direct learning from verifier rejections fails due to sparse data while fixed ontology and evidence-support filters succeed, with selectivity determined by matchi...

  23. Entity Binding Failures in Tool-Augmented Agents

    cs.AI 2026-06 unverdicted novelty 7.0

    The paper defines entity binding failures as a distinct error category in tool-augmented agents separate from tool selection errors and evaluates entity-aware mechanisms that eliminate such failures in a controlled di...

  24. CLQT: A Closed-Loop, Cost-Aware, Strategy-Consistent Benchmark for Diagnostic Evaluation of LLM Portfolio-Management Agents

    cs.AI 2026-06 unverdicted novelty 7.0

    CLQT is a new closed-loop, cost-aware benchmark that diagnoses LLM trading agent capabilities through strategy-consistent metrics and hash-verifiable trails rather than outcome rankings.

  25. A hardware-safety-gated system for LLM-written native ARTIQ control code on a trapped-ion platform

    quant-ph 2026-06 unverdicted novelty 7.0

    A token-based authorization system with simulation and human gates enables safe LLM-written ARTIQ control code execution on trapped-ion platforms while blocking unauthorized hardware access.

  26. Toward Agentic SysAdmin: Rethinking System Administration with AI Agents

    cs.NI 2026-06 unverdicted novelty 7.0

    NetLLMeval is an emulation-based framework for benchmarking LLM solvers on network admin tasks, with a 24000-run study showing solver architecture lifts a 14B model from 0.43 to 0.88 accuracy and allows local models t...

  27. Optimizing CUDA like a Human: Micro-Profiling Tools as Expert Surrogates for LLM-Based GPU Kernel Optimization

    cs.LG 2026-06 conditional novelty 7.0

    KernelPro combines LLM code generation, roofline-guided tool orchestration, and domain-adapted MCTS to produce GPU kernels that outperform prior automated and some hand-tuned baselines on KernelBench and VeOmni workloads.

  28. The Web4 Agent Economy: A Large-Scale Empirical Study of the Landscape, Challenges, and Opportunities

    cs.SE 2026-06 unverdicted novelty 7.0

    Empirical study of 99k multi-chain identities, 317M transactions, 341 MCP projects and 349 issues finds active daily M2M payments but immature infrastructure in identity, cross-environment operation and payment intero...

  29. Beyond Next-Observation Prediction: Agent-Authored World Modeling for Sequential Decision Making

    cs.CL 2026-06 unverdicted novelty 7.0

    AAWM builds training targets for world models by retrieving and synthesizing transition evidence based on the policy's self-identified decision needs at each state.

  30. FirmCure:Towards Autonomous and Adaptive Rehosting of Linux-Based Firmware

    cs.CR 2026-06 unverdicted novelty 7.0

    FirmCure uses LLM-based adaptive perception, reflective synthesis, and autonomous runtime intervention to rehost 21 firmware images at 100% port opening and 90.5% interactivity rates across 5 architectures.

  31. RS-Gen: A Multi-Stage Agentic Framework for Reasoning and Search-Augmented Image Generation

    cs.CV 2026-06 unverdicted novelty 7.0

    RS-Gen proposes a plug-and-play agentic framework with a closed-loop reasoning mechanism that augments base image models to achieve SOTA results on WISE Verified and RISEBench.

  32. Plans Don't Persist: Why Context Management Is Load Bearing for LLM Agents

    cs.AI 2026-06 unverdicted novelty 7.0

    Replay pairing shows LLM agents do not persist plans in hidden states but rely on plans remaining in context, with rapid signal decay and task performance drops when plans are evicted.

  33. When Agents Commit Too Soon: Diagnosing Premature Commitment in LLM Agents

    cs.AI 2026-06 unverdicted novelty 7.0

    Hidden-state convergence at step 4 predicts behavioral consistency in LLM agents on QA tasks (r=-0.35 to -0.83), enabling AUROC 0.97 detection of inconsistent trajectories but not improving accuracy on harder benchmarks.

  34. Lost in Aggregation: A Multi-Scale Diagnostic Benchmark for LLM Spatial Navigation

    physics.soc-ph 2026-06 unverdicted novelty 7.0

    A new diagnostic benchmark decomposes LLM spatial navigation into three cognitive scales and shows that cross-scale aggregation, not single-level deficits, causes failure beyond small mazes.

  35. PowerAgentBench-Dyn: A Benchmark for Agentic AI in Power System Dynamic Studies

    eess.SY 2026-06 unverdicted novelty 7.0

    PowerAgentBench-Dyn proposes a benchmark for evaluating agentic AI on dynamic model quality review and security risk screening tasks in power systems.

  36. ENPIRE: Agentic Robot Policy Self-Improvement in the Real World

    cs.AI 2026-06 unverdicted novelty 7.0

    ENPIRE supplies four modules (Environment, Policy Improvement, Rollout, Evolution) that turn real-world robot training into an autonomous optimization loop driven by coding agents.

  37. SIGMA: Skill-Incidence Graphs for Compositional Multi-Agent Design

    cs.MA 2026-06 unverdicted novelty 7.0

    SIGMA introduces skill-incidence graphs to compose agents from reusable skills, yielding higher average performance and robustness than topology-only baselines on reasoning and coding benchmarks.

  38. SAGE-OPD: Selective Agent-Guided Intervention for Multi-Turn On-Policy Distillation

    cs.CL 2026-06 unverdicted novelty 7.0

    SAGE-OPD improves multi-turn OPD via turn-level selective intervention, teacher-confidence weighting, and loss normalization, reporting up to 13.3% relative gain in ALFWorld unseen success rate over standard OPD.

  39. GraphPO: Graph-based Policy Optimization for Reasoning Models

    cs.CL 2026-06 unverdicted novelty 7.0

    GraphPO represents reasoning rollouts as a DAG to merge semantically equivalent paths, share suffixes, and assign separate efficiency and correctness advantages for lower variance and better performance than chain or ...

  40. Deep Research in Physical Sciences: A Multi-Agent Framework and Comprehensive Benchmark

    physics.comp-ph 2026-06 unverdicted novelty 7.0

    PhySciBench benchmark shows current AI models achieve at most 33.5% accuracy on physical science tasks; DelveAgent framework improves accuracy by up to 7.5 points and cuts costs to one-third.

  41. The Gate Is Only as Honest as Its Contracts: ContractGuard for the Contract Layer of Risk-Aware Causal Gating

    cs.CR 2026-06 unverdicted novelty 7.0

    ContractGuard verifies tool contracts in RACG systems to prevent effect forgery, restoring zero injection success on benchmarks and six hosted models against adaptive attackers.

  42. Verified Detection and Prevention of Concurrency Anomalies in Multi-Agent Large Language Model Systems

    cs.LG 2026-06 accept novelty 7.0

    Formalizes four concurrency anomalies in multi-agent LLM systems and mechanically verifies a hierarchy of sound detectors and preventions realized in Rust runtimes using TLA+ and Verus.

  43. Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents

    cs.CR 2026-06 unverdicted novelty 7.0

    Introduces a stakeholder-centric benchmark showing current web agents fail all tested prompt injection objectives, with failures falling into stealthy parasitism, misaligned disruption, or compounded failure modes.

  44. SENTINEL: Failure-Driven Reinforcement Learning for Training Tool-Using Language Model Agents

    cs.CL 2026-06 unverdicted novelty 7.0

    SENTINEL generates targeted tasks from model failures in a Controller-Proposer-Solver loop, raising Pass^1 from 66.4 to 74.9 on Tau2-Bench Retail and outperforming standard RL.

  45. GeoNatureAgent Benchmark: Benchmarking LLM Agents for Environmental Geospatial Analysis Across Frontier and Open-Weight Foundation Models

    cs.AI 2026-06 accept novelty 7.0

    GeoNatureAgent Benchmark tests seven LLMs on 93 tasks via a production geospatial API, with Claude Sonnet 4 at 60.8% and DeepSeek V3.2 offering near performance at 11x lower cost while all models fail on close-value c...

  46. OCELOT: Inference-Leakage Budgets for Privacy-Preserving LLM Agents

    cs.CR 2026-06 unverdicted novelty 7.0

    OCELOT recasts agent privacy as posterior-risk control and implements Witness-Verified Declassification to authorize the least-disclosing useful release under a sink-trust-weighted min-entropy budget.

  47. FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents

    cs.CL 2026-06 unverdicted novelty 7.0

    FORT synthesizes shortcut-resistant search tasks by controlling four identified shortcut risks across entity selection, graph construction, question formulation, and refinement, producing training data that yields age...

  48. Layer-Isolated Evaluation: Gating the Deterministic Scaffold of a Production LLM Agent with a No-LLM, Regression-Locked Test Harness

    cs.CL 2026-06 accept novelty 7.0

    Layer-isolated evaluation decomposes LLM agents into per-layer deterministic no-LLM test slices whose locked baselines localize regressions that aggregate pass rates mask.

  49. Self-Harness: Harnesses That Improve Themselves

    cs.CL 2026-06 unverdicted novelty 7.0

    Self-Harness lets LLM agents autonomously refine their interaction harnesses through weakness mining, proposal generation, and validation, raising held-out pass rates on Terminal-Bench-2.0 from 40.5% to 61.9%, 23.8% t...

  50. Co-Evolving Skill Generation and Policy Optimization

    cs.CL 2026-06 unverdicted novelty 7.0

    Framework estimates context-dependent marginal utility of candidate skills via reward gaps in matched base vs. skill-augmented rollouts to filter skills and co-train policy as generator.

  51. DICE: Entropy-Regularized Equilibrium Selection for Stable Multi-Agent LLM Coordination

    cs.LG 2026-06 unverdicted novelty 7.0

    DICE formalizes multi-agent LLM coordination as discounted incomplete-information Markov games and introduces Heterogeneous Quantal Response Equilibrium (HQRE) to achieve unique stable equilibria with bounded regret, ...

  52. Beyond Goodhart's Law: A Dynamic Benchmark for Evaluating Compliance in Multi-Agent Systems

    cs.AI 2026-06 unverdicted novelty 7.0

    MAC-Bench is a new adversarial benchmark that converts legal texts into executable scenarios via the SERV pipeline to measure procedural compliance in multi-agent LLM systems using CSR and MG metrics.

  53. Lightfall: An API-first, LLM-addressable control platform for synchrotron beamlines

    physics.ins-det 2026-06 unverdicted novelty 7.0

    Lightfall provides an API-first, LLM-addressable control platform for synchrotron beamlines enabling natural language extensions by scientists through plugin skills.

  54. ADK Arena: Evaluating Agent Development Kits via LLM-as-a-Developer

    cs.SE 2026-06 unverdicted novelty 7.0

    ADK Arena evaluates 51 Python ADKs by having an LLM learn each framework's API, write and repair agent code, and run on benchmarks, finding 57% success rate, 5.6x cost variation, no dominant framework, and substitutab...

  55. VASO: Formally Verifiable Self-Evolving Skills for Physical AI Agents

    cs.RO 2026-06 unverdicted novelty 7.0

    VASO is a verification-guided self-evolution framework for LLM robot skill contracts that reaches 97.2% formal-specification compliance on Jackal and quadcopter tasks using under 100 samples.

  56. AIP: A Graph Representation for Learning and Governing Agent Skills

    cs.AI 2026-06 unverdicted novelty 7.0

    AIP models skills as graphs of discrete steps connected by typed I/O edges under a validated schema, raising agent mean reward from 0.60 to 0.71 and pass rate from 53% to 67% on 27 SkillsBench tasks while enabling nod...

  57. PersonaTree: Structured Lifecycle Memory for Person Understanding in LLM Agents

    cs.CL 2026-06 unverdicted novelty 7.0

    PersonaTree is a new hierarchical memory framework for persistent LLM agents that structures evidence into persona claims via support paths and outperforms baselines on six person-understanding benchmarks.

  58. RealClawBench: Live OpenClaw Benchmarks from Real Developer-Agent Sessions

    cs.CL 2026-06 unverdicted novelty 7.0

    RealClawBench turns 281 real OpenClaw sessions into reproducible tasks that preserve the original distribution and shows the best of 14 models solves only 65.8 percent.

  59. Testing LLM Arithmetic Reasoning Generalization with Automatic Numeric-Remapping Attacks

    cs.CR 2026-06 unverdicted novelty 7.0

    An automatic numeric-remapping attack generator reveals 12-26 point accuracy drops on GSM8K for three LLMs while MAWPS and MultiArith stay near 98%.

  60. AUDITFLOW: Executable Symbolic Environments for Structured Financial Reporting Verification

    cs.AI 2026-06 unverdicted novelty 7.0

    AuditFlow combines a graph-grounded symbolic environment with a multi-agent LLM setup to reach 82.09% joint audit accuracy on structured financial reports, 14.93 points above the strongest baseline.