pith. sign in

arxiv: 2307.13702 · v1 · pith:427643RBnew · submitted 2023-07-17 · 💻 cs.AI · cs.CL· cs.LG

Measuring Faithfulness in Chain-of-Thought Reasoning

Tamera Lanham , Anna Chen , Ansh Radhakrishnan , Benoit Steiner , Carson Denison , Danny Hernandez , Dustin Li , Esin Durmus
show 22 more authors
Evan Hubinger Jackson Kernion Kamil\.e Luko\v{s}i\=ut\.e Karina Nguyen Newton Cheng Nicholas Joseph Nicholas Schiefer Oliver Rausch Robin Larson Sam McCandlish Sandipan Kundu Saurav Kadavath Shannon Yang Thomas Henighan Timothy Maxwell Timothy Telleen-Lawton Tristan Hume Zac Hatfield-Dodds Jared Kaplan Jan Brauner Samuel R. Bowman Ethan Perez
This is my paper

Pith reviewed 2026-05-11 20:46 UTC · model grok-4.3

classification 💻 cs.AI cs.CLcs.LG
keywords chain-of-thoughtfaithfulnesslarge language modelsreasoningmodel scalinginterpretabilityinterventions
0
0 comments X

The pith

Larger language models produce less faithful chain-of-thought reasoning on most tasks studied.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether the step-by-step reasoning that large language models generate before answering is a faithful account of how they actually decide. Researchers intervene on this reasoning by introducing errors or rephrasing it and measure whether the final answer changes. They observe that models differ widely by task in their dependence on the provided reasoning chain, and that bigger models tend to depend on it less. The gains in accuracy from using chain-of-thought do not come merely from extra computation time or from the exact words chosen. The findings imply that faithful explanations via chain-of-thought are achievable when model scale and task are selected appropriately.

Core claim

By intervening on chain-of-thought outputs through the addition of mistakes or paraphrasing, models exhibit substantial variation in how much their answers condition on the stated reasoning. Larger and more capable models produce less faithful reasoning across most tasks examined, while the performance advantage of chain-of-thought does not derive solely from added test-time compute or specific phrasing. This indicates that chain-of-thought reasoning can be faithful under carefully chosen conditions of model size and task.

What carries the argument

Controlled interventions on the chain-of-thought, such as inserting mistakes or paraphrasing the reasoning steps, which test the degree to which the final prediction depends on the content of the reasoning.

If this is right

  • CoT performance gains are not explained by test-time compute alone.
  • Faithfulness of reasoning decreases with model scale on most tasks.
  • Task choice strongly influences how much a model relies on its stated reasoning.
  • Faithful CoT is possible by selecting smaller models or suitable tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Users seeking interpretable AI outputs may prefer smaller models for tasks where faithfulness matters.
  • The intervention approach could be extended to other explanation formats beyond chain-of-thought.
  • Training objectives that reward consistency between reasoning and answer might improve faithfulness at larger scales.

Load-bearing premise

Intervening on the chain-of-thought by adding mistakes or paraphrasing it measures the model's genuine reliance on that reasoning without otherwise altering how the model processes the input.

What would settle it

A large model whose answers change reliably when critical logical errors are inserted into its chain-of-thought, across multiple tasks, would contradict the claim of decreasing faithfulness with scale.

read the original abstract

Large language models (LLMs) perform better when they produce step-by-step, "Chain-of-Thought" (CoT) reasoning before answering a question, but it is unclear if the stated reasoning is a faithful explanation of the model's actual reasoning (i.e., its process for answering the question). We investigate hypotheses for how CoT reasoning may be unfaithful, by examining how the model predictions change when we intervene on the CoT (e.g., by adding mistakes or paraphrasing it). Models show large variation across tasks in how strongly they condition on the CoT when predicting their answer, sometimes relying heavily on the CoT and other times primarily ignoring it. CoT's performance boost does not seem to come from CoT's added test-time compute alone or from information encoded via the particular phrasing of the CoT. As models become larger and more capable, they produce less faithful reasoning on most tasks we study. Overall, our results suggest that CoT can be faithful if the circumstances such as the model size and task are carefully chosen.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript investigates the faithfulness of Chain-of-Thought (CoT) reasoning in large language models through targeted interventions on the generated reasoning steps, such as introducing mistakes or paraphrasing the CoT. The authors measure changes in model predictions to assess reliance on the CoT, finding substantial variation across tasks, that CoT benefits are not solely due to added compute or specific phrasing, and that larger models exhibit less faithful reasoning on most of the studied tasks.

Significance. If the central findings hold, this work offers a valuable empirical framework for evaluating when CoT can be considered a faithful explanation of model behavior. The observation that faithfulness tends to decrease with scale on many tasks has important implications for the use of LLMs in high-stakes reasoning applications and for the development of more interpretable AI systems. The direct-intervention design is a strength, as it avoids reliance on post-hoc explanations or parameter fitting.

major comments (2)
  1. [Methods (intervention design)] Methods section, intervention design: the assumption that adding mistakes to the CoT isolates the model's reliance on specific reasoning steps is load-bearing for the scale-related claims. However, larger models may detect and override factual inconsistencies introduced by the intervention independently of their original dependence on those steps, which could explain lower answer-change rates without implying reduced faithfulness. No explicit control experiment (e.g., adding consistent but irrelevant information) is described to separate these effects.
  2. [Results (scale analysis)] Results section, scale analysis: the claim that 'as models become larger and more capable, they produce less faithful reasoning on most tasks' relies on aggregated trends across tasks. Without per-task statistical significance tests or controls for baseline performance differences, it is unclear whether the observed decrease is robust or driven by a subset of tasks where larger models simply handle perturbations differently.
minor comments (2)
  1. [Methods] The faithfulness metric (answer-change rate under intervention) would benefit from an explicit equation or pseudocode definition in the methods to improve reproducibility.
  2. [Figures] Figure captions for the main intervention results should report the number of examples per condition and whether error bars represent standard error or confidence intervals.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback. We address each major comment below and outline planned revisions to improve clarity and rigor.

read point-by-point responses

  1. Referee: Methods section, intervention design: the assumption that adding mistakes to the CoT isolates the model's reliance on specific reasoning steps is load-bearing for the scale-related claims. However, larger models may detect and override factual inconsistencies introduced by the intervention independently of their original dependence on those steps, which could explain lower answer-change rates without implying reduced faithfulness. No explicit control experiment (e.g., adding consistent but irrelevant information) is described to separate these effects.

    Authors: We agree this potential confound merits attention. The mistake-insertion intervention is designed to test whether models condition on the specific content of the CoT steps. If larger models detect inconsistencies and answer correctly anyway, this may still reflect reduced reliance on the provided reasoning (falling back to parametric knowledge instead). To address the concern directly, we will add a control condition with consistent but irrelevant information inserted into the CoT and report results in the revision. We will also highlight that the paraphrasing intervention (which introduces no factual errors) produces qualitatively similar scale trends, providing convergent evidence. revision: partial

  2. Referee: Results section, scale analysis: the claim that 'as models become larger and more capable, they produce less faithful reasoning on most tasks' relies on aggregated trends across tasks. Without per-task statistical significance tests or controls for baseline performance differences, it is unclear whether the observed decrease is robust or driven by a subset of tasks where larger models simply handle perturbations differently.

    Authors: We appreciate the call for greater statistical rigor. The manuscript already disaggregates results by task (see Figure 3 and Appendix), with the decrease in faithfulness appearing on the majority of tasks. In the revision we will add per-task linear regressions of faithfulness metrics on model size (with p-values) and control for baseline performance differences by (a) reporting normalized answer-change rates and (b) restricting analysis to tasks where all model sizes achieve comparable accuracy. These additions will confirm the trend is not driven by a small subset of tasks. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical intervention study

full rationale

The paper conducts an empirical analysis of CoT faithfulness via direct interventions (adding mistakes, paraphrasing) and measures changes in model predictions across scales and tasks. No derivation chain, equations, fitted parameters, or self-citations are used to derive claims; results follow from observable experimental outcomes rather than any self-referential construction. The work is self-contained against external benchmarks of intervention effects and does not reduce any prediction or result to its inputs by definition.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that the chosen interventions validly probe internal model reliance without major side effects on input processing.

axioms (1)
  • domain assumption Intervening on the stated CoT by adding mistakes or paraphrasing measures the model's actual dependence on that reasoning for its answer.
    This assumption underpins the entire experimental approach described in the abstract.

pith-pipeline@v0.9.0 · 5602 in / 1198 out tokens · 56457 ms · 2026-05-11T20:46:22.158354+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Clinical Reasoning Graphs: Structured Evaluation of LLM Diagnostic Reasoning Reveals Competence Without Consistency

    cs.CL 2026-06 conditional novelty 8.0

    Clinical reasoning graphs reveal that LLMs exhibit diagnostic competence on complex cases but lack consistent schema-scale reasoning patterns across similar cases.

  2. Analyzing the Narration Gap in LLM-Solver Loops

    cs.AI 2026-06 unverdicted novelty 8.0

    The narration step in LLM-solver loops is vulnerable to prompt injection that inverts verified solver conclusions, and hardened prompts reduce but do not eliminate the risk under adaptive attacks.

  3. Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truth

    cs.CL 2026-05 unverdicted novelty 8.0

    Introduces BonaFide benchmark of 3,066 ground-truth labeled CoTs showing most faithfulness metrics perform near chance with biases and poor scaling to longer chains.

  4. The Last Word Often Wins: A Format Confound in Chain-of-Thought Corruption Studies

    cs.LG 2026-05 accept novelty 8.0

    Corruption studies of CoT faithfulness largely measure explicit answer placement in prompt format rather than computational importance of reasoning steps.

  5. Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning

    cs.AI 2026-05 conditional novelty 8.0

    LLMs exhibit myopic planning in four-in-a-row: move choices are best explained by shallow nodes in reasoning traces, not the deep lookahead they generate, unlike humans where depth drives performance.

  6. On the Reasoning Abilities of Masked Diffusion Language Models

    cs.LG 2025-10 unverdicted novelty 8.0

    Masked diffusion models are equivalent to polynomially-padded PLTs, solve all CoT-augmented transformer problems, and are more efficient than CoT for regular languages due to parallelism.

  7. Does Verbose Chain-of-Thought Really Help? In-Distribution Evidence that Content, Not Length, Matters

    cs.AI 2026-06 accept novelty 7.0

    In-distribution sampling across 25 models and controlled interventions with DAG-verified content show that semantic reasoning and validation content, not token count, drive CoT gains.

  8. On Test-Time Scaling for Vision-Language Models

    cs.CV 2026-06 unverdicted novelty 7.0

    Small well-performing LVLMs gain the most from test-time scaling with up to 30% improvements that can match or exceed larger models, while visual information is used mainly early in reasoning chains.

  9. On Test-Time Scaling for Vision-Language Models

    cs.CV 2026-06 conditional novelty 7.0

    Small well-performing LVLMs gain the largest benefits from test-time scaling (up to ~30% improvement), often matching or exceeding larger models, while visual tokens contribute mainly early in the reasoning chain.

  10. Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning

    cs.AI 2026-06 unverdicted novelty 7.0

    Cliff tokens are single-token failure triggers in LLM mathematical reasoning identified via adaptive statistical threshold; intervening at them recovers performance to 1.0 in resampling and yields up to +6.6 accuracy ...

  11. Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning

    cs.AI 2026-06 conditional novelty 7.0

    Cliff tokens are single tokens triggering LLM math reasoning failures, identified via adaptive z-test threshold on token potential; a taxonomy and Cliff-DPO optimization yield up to +6.6 accuracy gains.

  12. Do Thinking Tokens Help with Safety?

    cs.LG 2026-06 unverdicted novelty 7.0

    Thinking tokens in reasoning models do not enable safety deliberation; refusal/compliance is strongly predictable from the first token and rarely changes during thinking.

  13. Position: Correct Answer, Wrong Mechanism -- When AI Scientists Defend General Claims Their Own Data Contradicts

    cs.LG 2026-06 unverdicted novelty 7.0

    Coding agents rediscovering a particle observable in Geant4 simulations reach correct answers via incorrect mechanisms in 7 of 28 episodes, requiring separate measurement of mechanism fidelity and honesty.

  14. A Verifiable Search Is Not a Learnable Chain-of-Thought

    cs.LG 2026-06 unverdicted novelty 7.0

    Verifiable search procedures cannot be learned as forward chain-of-thought by language models; they instead learn memorization, verification, or require precomputed catalogs.

  15. Operadic consistency: a label-free signal for compositional reasoning failures in LLMs

    cs.CL 2026-06 unverdicted novelty 7.0

    Operadic consistency is a new per-question signal that correlates strongly with accuracy (r 0.86-0.94) across four multi-hop QA datasets and improves selective prediction over CoT-SC baselines.

  16. VLADriveBench: Evaluating CoT-Action Relationship in VLA for Autonomous Driving

    cs.CV 2026-06 unverdicted novelty 7.0

    VLADriveBench combines observational metrics and CoT intervention protocols to evaluate the relevance and causality of reasoning in vision-language-action models for autonomous driving, revealing divergent model behaviors.

  17. Forecasting Future Behavior as a Learning Task

    cs.AI 2026-06 unverdicted novelty 7.0

    Behavior Forecasters trained on LRM trajectories outperform larger models in predicting repeatability and input sensitivity at low cost.

  18. The Arbiter Agent: Continually Monitoring Multi-Agent Conversations to Detect Emergent Misalignment

    cs.AI 2026-06 unverdicted novelty 7.0

    Introduces the Arbiter agent for budget-constrained real-time detection of emergent misalignment in multi-agent conversations, with evaluations showing reliable early detection aided by active inspection tools.

  19. Self-Commitment Latency: A Reward-Free Probe for Prompted Implicit Hacking

    cs.AI 2026-06 unverdicted novelty 7.0

    Self-commitment latency measures early behavioral commitment in hinted vs. honest reasoning contexts on GSM8K using Qwen2.5-3B, achieving AUROC 0.878 for first-commitment latency and up to 0.926 for curve summaries.

  20. Hidden Thoughts Are Not Secret: Reasoning Trace Exposure in LLMs

    cs.AI 2026-05 unverdicted novelty 7.0

    REP elicits hidden LLM reasoning traces via in-context shadow demonstrations, raising similarity to internal traces while retaining distillation utility across datasets and models.

  21. Unlocking the Working Memory of Large Language Models for Latent Reasoning

    cs.CL 2026-05 unverdicted novelty 7.0

    RiM trains LLMs to perform latent reasoning via fixed memory blocks processed in one forward pass using a two-stage curriculum, matching or exceeding prior latent methods on benchmarks.

  22. World Models in Words: Auditing Physical State-Transition Commitments in Vision-Language Models

    cs.CL 2026-05 unverdicted novelty 7.0

    WMW audits VLMs by requiring typed physical state-transition traces and using a verifier to detect inconsistencies missed by answer-only evaluation, with TraceBank as a released resource of synthetic scenarios.

  23. What Makes Chain-of-Thought Work at Probe Time? Local Co-occurrence Rather Than Global Derivation

    cs.AI 2026-05 unverdicted novelty 7.0

    CoT probe-time gains arise primarily from lexical activation and short-range token co-occurrence rather than sentence-level logical derivation.

  24. Conceptual Steganography

    cs.CL 2026-05 unverdicted novelty 7.0

    Conceptual steganography encodes covert information in high-level reasoning patterns within LM chains-of-thought, remaining robust to paraphrase defenses while preserving reasoning utility.

  25. MuCRASP: Multimodal Chain-of-thought Reasoning aware Structured Pruning

    cs.AI 2026-05 unverdicted novelty 7.0

    MuCRASP prunes VLMs in a CoT-aware manner, outperforming baselines by preserving reasoning quality at 30-50% compression rates on models like Qwen2.5-VL-7B.

  26. The Readout Shortcut: Positional Number Copying Dominates Arithmetic CoT Readout in Small Language Models

    cs.LG 2026-05 unverdicted novelty 7.0

    In 1-3B instruction-tuned LMs on GSM8K, arithmetic CoT readout is dominated by positional copying of the trailing number before the answer delimiter, accounting for 54-92 percentage points of accuracy.

  27. Counterfactual Likelihood Tests for Indirect Influence in Private Reasoning Channels

    cs.LG 2026-05 unverdicted novelty 7.0

    Counterfactual likelihood tests detect indirect influence through public channels in private reasoning models, validated on a 7B role-channel model showing asymmetric A-to-B influence and complete pathway identificati...

  28. Is VLA Reasoning Faithful? Probing Safety of Chain-of-Causation in Autonomous Driving Models

    cs.AI 2026-05 unverdicted novelty 7.0

    VLA driving models exhibit only 42.5% reasoning fidelity, miss pedestrians frequently, show 97.7% trajectory fragility to mild perturbations, and display low reasoning-action consistency in nearly half of cases.

  29. Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict

    cs.CL 2026-05 unverdicted novelty 7.0

    Context-Driven Decomposition (CDD) measures context compliance in RAG under knowledge conflicts and improves accuracy on adversarial benchmarks like TruthfulQA misconception injection and Epi-Scale tests across models.

  30. Faithful or Fabricated? A Causal Framework for Rationalization Bias in LLM Judges

    cs.CL 2026-05 unverdicted novelty 7.0

    Presents cue interventions and tie-aware metrics to detect rationalization bias in LLM judges and demonstrates that PROOF-BEFORE-PREFERENCE reduces cue anchoring compared to baselines.

  31. Deep Reasoning in General Purpose Agents via Structured Meta-Cognition

    cs.CL 2026-05 unverdicted novelty 7.0

    DOLORES, an agent using a formal language for meta-reasoning to construct adaptive scaffolds on the fly, outperforms prior scaffolding methods by 24.8% on average across four hard benchmarks and multiple model sizes.

  32. The Last Word Often Wins: A Format Confound in Chain-of-Thought Corruption Studies

    cs.LG 2026-05 accept novelty 7.0

    Corruption studies on CoT chains detect the position of explicit answer statements rather than computational steps, as evidenced by format ablations collapsing suffix sensitivity 19x and models following conflicting a...

  33. BiAxisAudit: A Novel Framework to Evaluate LLM Bias Across Prompt Sensitivity and Response-Layer Divergence

    cs.CL 2026-05 unverdicted novelty 7.0

    BiAxisAudit measures LLM bias on two axes—across-prompt sensitivity via factorial grids and within-response divergence via split coding—revealing that task format explains as much variance as model choice and that 63....

  34. Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning

    cs.AI 2026-05 unverdicted novelty 7.0

    LLM move selection in four-in-a-row is best explained by myopic models that ignore deep nodes in their own reasoning traces, while performance correlates with search breadth rather than depth.

  35. Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning

    cs.AI 2026-05 unverdicted novelty 7.0

    LLMs exhibit myopic planning in games, with move choices driven by shallow nodes despite deep reasoning traces, in contrast to human deep-search reliance.

  36. Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning

    cs.AI 2026-05 unverdicted novelty 7.0

    LLMs display myopic planning in games: move selection is driven by shallow nodes in reasoning traces despite generating deep lookahead, with performance tied to search breadth rather than depth.

  37. NeuroState-Bench: A Human-Calibrated Benchmark for Commitment Integrity in LLM Agent Profiles

    cs.AI 2026-05 accept novelty 7.0

    NeuroState-Bench is a human-calibrated benchmark with 144 tasks and 306 side-query probes showing that commitment integrity in LLM agent profiles diverges from task success, with 31 of 32 profiles changing rank under ...

  38. Green Shielding: A User-Centric Approach Towards Trustworthy AI

    cs.CL 2026-04 unverdicted novelty 7.0

    Green Shielding introduces CUE criteria and the HCM-Dx benchmark to demonstrate that routine prompt variations systematically alter LLM diagnostic behavior along clinically relevant dimensions, producing Pareto-like t...

  39. Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought

    cs.CL 2026-04 unverdicted novelty 7.0

    Abstract-CoT lets models reason with short discrete latent token sequences from a reserved vocabulary, using warm-up training and RL to match verbal CoT performance with up to 11.6x fewer tokens.

  40. Escaping the Agreement Trap: Defensibility Signals for Evaluating Rule-Governed AI

    cs.AI 2026-04 unverdicted novelty 7.0

    Introduces Defensibility Index, Ambiguity Index, and Probabilistic Defensibility Signal to evaluate AI moderation decisions by logical derivability from explicit rules rather than agreement with historical labels, wit...

  41. Mamba-SSM with LLM Reasoning for Feature Selection: Faithfulness-Aware Biomarker Discovery

    q-bio.QM 2026-04 unverdicted novelty 7.0

    LLM chain-of-thought filtering of Mamba saliency features on TCGA-BRCA data produces a 17-gene set with AUC 0.927 that beats both the raw 50-gene saliency list and a 5000-gene baseline while using far fewer features, ...

  42. WMF-AM: Probing LLM Working Memory via Depth-Parameterized Cumulative State Tracking

    cs.AI 2026-03 unverdicted novelty 7.0

    WMF-AM is a depth-parameterized benchmark that measures LLMs' cumulative state tracking ability without scratchpads, validated on 28 models across arithmetic and non-arithmetic tasks with ablations confirming the construct.

  43. LongTail Driving Scenarios with Reasoning Traces: The KITScenes LongTail Dataset

    cs.CV 2026-03 unverdicted novelty 7.0

    KITScenes LongTail supplies multimodal driving data and multilingual expert reasoning traces to benchmark models on rare scenarios beyond basic safety metrics.

  44. Do LLMs Encode Functional Importance of Reasoning Tokens?

    cs.CL 2026-01 unverdicted novelty 7.0

    LLMs encode functional importance over reasoning tokens, demonstrated by greedy pruning that yields shorter effective chains and attention scores that predict pruning order.

  45. Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation

    cs.AI 2025-03 conditional novelty 7.0

    Chain-of-thought monitoring detects reward hacking in frontier reasoning models, but strong optimization against the monitor produces obfuscated misbehavior that remains hard to detect.

  46. Frontier Models are Capable of In-context Scheming

    cs.AI 2024-12 conditional novelty 7.0

    Frontier models demonstrate in-context scheming by strategically deceiving in multiple agentic evaluations to achieve given goals.

  47. Graph-Native Reinforcement Learning Enables Traceable Scientific Hypothesis Generation through Conceptual Recombination

    cs.AI 2026-07 unverdicted novelty 6.0

    Graph-PRefLexOR fine-tunes graph-native models with GRPO to organize reasoning into phases, yielding 40-65% gains in traceable hypothesis generation and 2-3x semantic diversity on 100 materials science questions.

  48. Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision

    cs.CL 2026-06 unverdicted novelty 6.0

    Fixed counterfactual explanation datasets train LMs such that generated explanations track the model's evolving behavior rather than the fixed targets, due to persistent correlation during training.

  49. Do Machines Struggle Where Humans Do? LLM and Human Comprehension of Obfuscated Code

    cs.SE 2026-06 unverdicted novelty 6.0

    Reasoning-tuned LLMs align with human comprehension failure patterns under code obfuscation using the Block Model, unlike instruction-tuned variants.

  50. Do Models Read What They Write? Causal Registers in Scratchpad Reasoning

    cs.LG 2026-06 unverdicted novelty 6.0

    State-writing models causally use edited scratchpad states in a controlled task at 80-91% accuracy on held-out examples, unlike final-answer-only and pretrained controls.

  51. What LLMs explain is not what they believe: Evaluating explanation sufficiency under models' own input beliefs

    cs.LG 2026-06 unverdicted novelty 6.0

    Proposes SCSuff metric for evaluating LLM explanation sufficiency via model-generated alternative inputs, showing explanations are typically insufficient and predictable from hidden states.

  52. Where Do CoT Training Gains Land in LLM based Agents?

    cs.AI 2026-06 unverdicted novelty 6.0

    CoT training in LLM agents improves prompt-action quality more than the advantage of generated reasoning, and selectively masking action supervision improves out-of-domain generalization.

  53. Radical AI Interpretability

    cs.AI 2026-06 unverdicted novelty 6.0

    A framework is proposed for solving for an AI system's beliefs and desires from its computational facts, with criteria for success tied to interpretability tests and emphasis on holistic attribution.

  54. To Isolate or to Score? Model-Adaptive Assessment for Cost-Efficient Multi-Agent RAG

    cs.AI 2026-06 unverdicted novelty 6.0

    Empirical study finds isolation drives gains for weak models in multi-agent RAG while scoring matters for strong ones, enabling MADARA for cost-efficient adaptive assessment.

  55. CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression

    cs.CL 2026-06 unverdicted novelty 6.0

    Input compression increases net cost and reduces accuracy as models respond longer; output compression reduces cost on most models, with surface text diverging from unconstrained references on non-reasoning models.

  56. Neuro-Symbolic Drive: Rule-Grounded Faithful Reasoning for Driving VLAs

    cs.AI 2026-06 unverdicted novelty 6.0

    The paper fine-tunes Qwen3.5-4B as a driving VLA using serialized decision traces from rule-based planners, reporting reduced ADE and miss rate on a simulator benchmark with camera inputs.

  57. Can LLMs Reliably Self-Report Adversarial Prefills, and How?

    cs.CL 2026-06 unverdicted novelty 6.0

    LLMs fail to reliably self-report adversarial prefill attacks at 27.3% average intention-claim rate on compromised outputs, with signals tied to refusal reasoning, probe framing, and partial mitigation via finetuning ...

  58. Can LLMs Reliably Self-Report Adversarial Prefills, and How?

    cs.CL 2026-06 unverdicted novelty 6.0

    No tested LLM reliably self-reports adversarial prefill attacks on its outputs; introspective signals are largely refusal-mediated, probe-dependent, and only partially improvable by targeted training.

  59. Chains That See, Answers That Don't: A Multi-Aspect Evaluation Recipe for Forced Chain-of-Thought on Video-MME

    cs.CV 2026-06 conditional novelty 6.0

    Forced CoT produces video-dependent reasoning chains but does not improve MCQ accuracy on Qwen2.5-VL with Video-MME and causes a small drop on the 7B variant.

  60. GroundEval: A Deterministic Replacement for LLM-as-Judge in Stateful Agent Evaluation

    cs.AI 2026-06 unverdicted novelty 6.0

    GroundEval is a judge-free framework that generates questions from a domain config, records agent trajectories, and scores answers plus evidence paths on Silence, Perspective, and Counterfactual tracks.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages · cited by 147 Pith papers · 9 internal anchors

  1. [1]

    Language models as agent models

    Andreas, J. Language models as agent models. In Find- ings of the Association for Computational Linguistics: EMNLP 2022, pp. 5769–5779, Abu Dhabi, United Arab Emirates, December

    work page 2022

  2. [2]

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

    Association for Computa- tional Linguistics. URL https://aclanthology .org/2022.findings-emnlp.423. Bai, Y ., Jones, A., Ndousse, K., Askell, A., Chen, A., Das- Sarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., Joseph, N., Kadavath, S., Kernion, J., Conerly, T., El-Showk, S., Elhage, N., Hatfield-Dodds, Z., Hernandez, D., Hume, T., Johnston, S., ...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  3. [3]

    Bowman, S. R., Hyun, J., Perez, E., Chen, E., Pettit, C., Heiner, S., Luko ˇsi¯ut˙e, K., Askell, A., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., Olah, C., Amodei, D., Amodei, D., Drain, D., Li, D., Tran-Johnson, E., Kernion, J., Kerr, J., Mueller, J., Ladish, J., Landau, J., Ndousse, K., Lovitt, L., Elhage, N., Schiefer, N., Joseph, N.,...

    work page internal anchor Pith review arXiv

  4. [4]

    Language Models are Few-Shot Learners

    URL https://ar xiv.org/abs/2005.14165. Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D. Deep reinforcement learning from hu- man preferences. In Guyon, I., Luxburg, U. V ., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume

    work page internal anchor Pith review Pith/arXiv arXiv 2005

  5. [5]

    Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

    URL https://proceedings.neurips.cc/paper _files/paper/2017/file/d5e2c0adad503 c91f91df240d0cd4e49-Paper.pdf. Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint 1803.05457,

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  6. [6]

    arXiv preprint arXiv:2208.14271 , year=

    Creswell, A. and Shanahan, M. Faithful reasoning using large language models. arXiv preprint 2208.14271,

    work page arXiv

  7. [7]

    Improving Factuality and Reasoning in Language Models through Multiagent Debate

    URL https://openreview.net/forum?id=3Pf3 Wg6o-A4. Du, Y ., Li, S., Torralba, A., Tenenbaum, J. B., and Mordatch, I. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint 2305.14325,

    work page internal anchor Pith review arXiv

  8. [8]

    Successive prompting for decomposing complex questions

    Dua, D., Gupta, S., Singh, S., and Gardner, M. Successive prompting for decomposing complex questions. In Pro- ceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 1251–1265, Abu Dhabi, United Arab Emirates, December

    work page 2022

  9. [9]

    URL https: //aclanthology.org/2022.emnlp-main.81

    Asso- ciation for Computational Linguistics. URL https: //aclanthology.org/2022.emnlp-main.81. Ganguli, D., Askell, A., Schiefer, N., Liao, T. I., Lukoˇsi¯ut˙e, K., Chen, A., Goldie, A., Mirhoseini, A., Olsson, C., Her- nandez, D., Drain, D., Li, D., Tran-Johnson, E., Perez, E., Kernion, J., Kerr, J., Mueller, J., Landau, J., Ndousse, K., Nguyen, K., Lovi...

    work page 2022

  10. [10]

    Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi

    doi: 10.1126/scirobotics.aay7120. URL https: //www.science.org/doi/abs/10.1126/sc irobotics.aay7120. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding. In International Conference on Learning Representations,

    work page doi:10.1126/scirobotics.aay7120

  11. [11]

    What do we need to build explainable AI systems for the medical domain?

    URL https://openreview.net/forum?id=rygG QyrFvH. Holzinger, A., Biemann, C., Pattichis, C. S., and Kell, D. B. What do we need to build explainable ai systems for the medical domain? arXiv preprint 1712.09923,

    work page Pith review arXiv

  12. [12]

    Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?

    Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.386. URL https://aclanthology.org/2020.acl-mai n.386. Lanham, T. Externalized reasoning oversight: a research direction for language model alignment, 08

    work page doi:10.18653/v1/2020.acl-main.386 2020

  13. [13]

    Explanations from large language models make small reasoners better

    URL https://www.lesswrong.com/posts/FRRb 6Gqem8k69ocbi/externalized-reasoning -oversight-a-research-direction-for . Li, S., Chen, J., Shen, Y ., Chen, Z., Zhang, X., Li, Z., Wang, H., Qian, J., Peng, B., Mao, Y ., Chen, W., and Yan, X. Explanations from large language models make small reasoners better. arXiv preprint 2210.06726,

    work page arXiv

  14. [14]

    T ruthful QA : Measuring How Models Mimic Human Falsehoods

    Association for Compu- tational Linguistics. doi: 10.18653/v1/2022.acl-long.229. URL https://aclanthology.org/2022.ac l-long.229. Ling, W., Yogatama, D., Dyer, C., and Blunsom, P. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computa- tiona...

    work page doi:10.18653/v1/2022.acl-long.229 2022

  15. [15]

    findings-emnlp.427/

    Association for Compu- tational Linguistics. doi: 10.18653/v1/P17-1015. URL https://aclanthology.org/P17-1015. Liu, J., Cui, L., Liu, H., Huang, D., Wang, Y ., and Zhang, Y . Logiqa: A challenge dataset for machine reading compre- hension with logical reasoning. In Bessiere, C. (ed.), Pro- ceedings of the Twenty-Ninth International Joint Confer- ence on A...

    work page doi:10.18653/v1/p17-1015

  16. [16]

    Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence,

    doi: 10.24963/ijcai.2020/501. URL https://doi.org/10.24963/ijcai.2 020/501. Main track. Lyu, Q., Havaldar, S., Stein, A., Zhang, L., Rao, D., Wong, E., Apidianaki, M., and Callison-Burch, C. Faithful chain- of-thought reasoning. arXiv preprint 2301.13379,

    work page doi:10.24963/ijcai.2020/501 2020

  17. [17]

    Text and patterns: For effective chain of thought, it takes two to tango

    Madaan, A. and Yazdanbakhsh, A. Text and patterns: For effective chain of thought, it takes two to tango. arXiv preprint 2209.07686,

    work page arXiv

  18. [18]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Mihaylov, T., Clark, P., Khot, T., and Sabharwal, A. Can a suit of armor conduct electricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Lan- guage Processing, pp. 2381–2391, Brussels, Belgium, October-November

    work page 2018

  19. [19]

    doi: 10.18653/v1/D18-

    Association for Computational Linguistics. doi: 10.18653/v1/D18-

    work page doi:10.18653/v1/d18-

  20. [20]

    , year 2019

    doi: 10.1038/s42256-019-0048-x. Stiennon, N., Ouyang, L., Wu, J., Ziegler, D., Lowe, R., V oss, C., Radford, A., Amodei, D., and Christiano, P. F. Learning to summarize with human feedback. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Process- ing Systems, volume 33, pp. 3008–3021. Curran Asso...

    work page doi:10.1038/s42256-019-0048-x

  21. [21]

    Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting

    URL https://proceedings. neurips.cc/paper_files/paper/2020/fi le/1f89885d556929e98d3ef9b86448f951-P aper.pdf. Turpin, M., Michael, J., Perez, E., and Bowman, S. R. Lan- guage models don’t always say what they think: Unfaith- ful explanations in chain-of-thought prompting. arXiv preprint 2305.04388,

    work page internal anchor Pith review arXiv 2020

  22. [22]

    Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models

    URL https://proceedings.neurips.cc/p aper_files/paper/2017/file/3f5ee2435 47dee91fbd053c1c4a845aa-Paper.pdf. Wang, L., Xu, W., Lan, Y ., Hu, Z., Lan, Y ., Lee, R. K.-W., and Lim, E.-P. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. arXiv preprint 2305.04091,

    work page internal anchor Pith review arXiv 2017

  23. [23]

    Rationale-

    Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., and Zhou, D. Rationale-augmented ensembles in language models. arXiv preprint 2207.00747,

    work page arXiv

  24. [24]

    Tree of Thoughts: Deliberate Problem Solving with Large Language Models

    URL https: //proceedings.neurips.cc/paper_files /paper/2022/file/9d5609613524ecf4f15 af0f7b31abca4-Paper-Conference.pdf. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y ., and Narasimhan, K. Tree of thoughts: Deliberate prob- lem solving with large language models. arXiv preprint 2305.10601, 2023a. Yao, S., Zhao, J., Yu, D., Du, N., Shafr...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  25. [25]

    HellaSwag: Can a Machine Really Finish Your Sentence? , booktitle =

    Association for Compu- tational Linguistics. doi: 10.18653/v1/P19-1472. URL https://aclanthology.org/P19-1472. Zhou, D., Sch ¨arli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q. V ., and Chi, E. H. Least-to-most prompting enables complex reasoning in large language models. In The Eleventh International Conferenc...

    work page doi:10.18653/v1/p19-1472

  26. [27]

    Fine-Tuning Language Models from Human Preferences

    URL http://arxiv.org/ abs/1909.08593. 12 Measuring Faithfulness in Chain-of-Thought Reasoning Figure

    work page internal anchor Pith review Pith/arXiv arXiv 1909