Pith. sign in

REVIEW 1 major objections 1 minor 1 cited by

Memory-layer tool gating stops persistent memory attacks on eight of nine LLM agents by blocking recall but inverts to full success on one reasoning model.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · grok-4.3

2026-06-30 22:52 UTC pith:UYQQEWLH

load-bearing objection Memory sandbox stops attacks on eight models but inverts to 100% ASR on one reasoning model, showing layer placement does not predict outcomes independently of model-specific paths. the 1 major comments →

arxiv 2605.08442 v3 pith:UYQQEWLH submitted 2026-05-08 cs.CR cs.AIcs.LG

Defense effectiveness across architectural layers: a mechanistic evaluation of persistent memory attacks on stateful LLM agents

classification cs.CR cs.AIcs.LG
keywords persistent memory attacksLLM agentsdefense evaluationarchitectural layersattack success rateMemory SandboxRAG
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests six defenses placed at four different layers of LLM agent architecture against persistent memory attacks that store malicious instructions for later execution. Input filtering and retrieval filtering leave attack success rates at baseline levels around 88 percent because they cannot observe or correctly flag the injected content. Prompt hardening shows partial reduction driven by model-specific refusal on two cases. Tool gating at the memory layer eliminates the recall step the attack needs and drives attack success to zero on eight models with no measured cost to normal task performance. One reasoning model that refuses attacks without any defense instead achieves full attack success under the memory defense because blocking recall shifts it to a retrieval pathway that bypasses its refusal mechanism.

Core claim

Persistent memory attacks succeed by storing malicious instructions through RAG documents for delayed execution; input and retrieval defenses fail because injected content remains invisible or semantically masked, while memory-layer tool gating removes the recall capability required for execution on eight of nine models and imposes zero utility cost on benign tasks, except when the defense forces a reasoning model onto an RAG pathway where its refusal does not activate.

What carries the argument

Memory Sandbox: a tool-gating defense at the memory layer that prevents recall and use of previously stored instructions.

Load-bearing premise

Persistent memory attacks depend on the model explicitly recalling stored content, and defense success is determined by architectural layer placement independent of model-specific reasoning pathways.

What would settle it

Measuring attack success rate on the inverting reasoning model when recall is blocked but the model is still given the option to process the malicious content through its normal refusal-triggering path.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Input-level and retrieval-level defenses achieve attack success rates of 88-89 percent, statistically identical to the undefended baseline.
  • Memory Sandbox reduces attack success to zero on eight models while maintaining 100 percent benign task completion rate.
  • Prompt hardening lowers overall attack success to 77.8 percent but only through refusal behavior on two specific models.
  • Architectural layer placement interacts with model reasoning style such that blocking recall can activate alternate vulnerable pathways.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Defenses may need to preserve or replicate refusal mechanisms across multiple information pathways rather than relying on single-layer intervention.
  • Evaluation of new defenses should include models that already exhibit strong refusal to detect cases where layer changes shift execution routes.
  • Layer-based defenses could be combined to close both recall and retrieval routes simultaneously.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. The paper reports a large-scale empirical study (5040 runs, N=40 per condition) evaluating six defenses across four architectural layers against persistent memory attacks on nine open-source LLM agents. It finds that input-level (Minimizer, Sanitizer) and retrieval-level (RAG Sanitizer, RAG LLM Judge) defenses achieve 88-89% ASR, statistically indistinguishable from the undefended 88.6% baseline; Prompt Hardening reaches 77.8% ASR with effects driven by two models; and Memory Sandbox at the memory layer reduces ASR to 0% for eight models by blocking explicit recall, but inverts to 100% ASR on one reasoning model (which had 0% undefended ASR via execution refusal) by forcing the RAG pathway where refusal does not activate. The work claims this provides the first systematic characterization of why each defense class fails, with Memory Sandbox imposing zero utility cost (BTCR=100%).

Significance. If the results hold, the large-scale direct comparison to baseline and the identification of an effective memory-layer defense with no utility penalty would provide actionable guidance for defense design against persistent memory attacks. The scale (5040 runs) and inclusion of an outlier case strengthen the empirical contribution, though the post-hoc interpretation of the inversion limits the strength of the architectural-layer claims.

major comments (1)
  1. [Abstract] Abstract: the claim that 'the architectural explanation holds' (input-level defenses cannot observe RAG content; retrieval classifiers defeated by masking) is challenged by the reported inversion under Memory Sandbox, where one reasoning model's success depends on its specific refusal pathway rather than layer placement; the post-hoc mechanistic story for the outlier lacks a pre-specified ablation or test confirming that inverted attacks route via RAG rather than another mechanism, undermining the assertion that layer placement independently predicts defense outcome across models.
minor comments (1)
  1. [Abstract] The abstract reports statistical indistinguishability from baseline but does not name the exact tests or correction methods used; adding these details would improve verifiability of the 'fail at approximately baseline' claims.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback on the abstract. We agree that the outlier case requires more careful qualification of the architectural claims and will revise the abstract accordingly.

read point-by-point responses
  1. Referee: [Abstract] Abstract: the claim that 'the architectural explanation holds' (input-level defenses cannot observe RAG content; retrieval classifiers defeated by masking) is challenged by the reported inversion under Memory Sandbox, where one reasoning model's success depends on its specific refusal pathway rather than layer placement; the post-hoc mechanistic story for the outlier lacks a pre-specified ablation or test confirming that inverted attacks route via RAG rather than another mechanism, undermining the assertion that layer placement independently predicts defense outcome across models.

    Authors: We accept this point. The inversion on the reasoning model (0% undefended ASR via execution refusal, 100% under Memory Sandbox) shows that model-specific refusal pathways can override layer-based predictions. Our explanation that Memory Sandbox forces the RAG pathway is inferred from the attack design and observed behaviors rather than from a pre-specified ablation isolating RAG vs. explicit recall. We will revise the abstract to state that the architectural explanation holds for eight of nine models, with one noted exception driven by model-specific mechanisms. This avoids claiming independent prediction across all models while preserving the empirical comparison of defense classes. revision: partial

Circularity Check

0 steps flagged

No circularity: purely empirical evaluation with direct measurements

full rationale

The paper reports results from 5,040 experimental runs measuring attack success rates (ASR) under different defenses across models. No equations, fitted parameters, derivations, or self-citations are used to support the central claims. The reported outcomes (e.g., Memory Sandbox reducing ASR to 0% for eight models, with one inversion) are direct observations from the test conditions, not quantities defined by or reduced to the inputs by construction. The analysis is self-contained against the external benchmark of measured ASR values.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper is an empirical evaluation study. No free parameters, mathematical axioms beyond standard statistics, or new postulated entities are introduced in the visible abstract.

pith-pipeline@v0.9.1-grok · 5845 in / 1188 out tokens · 33884 ms · 2026-06-30T22:52:43.343038+00:00 · methodology

0 comments
read the original abstract

Persistent memory attacks against LLM agents achieve high attack success rates against open-source models. In these attacks, malicious instructions injected via RAG-retrieved documents are stored in persistent memory and executed in later sessions. However, no systematic evaluation of defense effectiveness against this attack class exists. We evaluate six defenses across four architectural layers against delayed-trigger attacks on nine open-source models (5,040 runs, N=40 per condition). Four defenses fail at approximately baseline attack success rate: input-level filtering (Minimizer, Sanitizer) and retrieval-level filtering (RAG Sanitizer, RAG LLM Judge) achieve 88-89% ASR, statistically indistinguishable from the undefended baseline of 88.6%. Prompt Hardening partially fails at 77.8% ASR, with the reduction driven by two models at 0%: one genuine defense effect and one model-level refusal independent of the defense. The architectural explanation holds: input-level defenses cannot observe RAG-injected content, and retrieval-level classifiers are defeated by compliance-framed semantic masking. One defense, tool-gating at the memory layer (Memory Sandbox), reduces ASR to 0% for eight of nine models by removing the recall capability the attack requires. The exception inverts the defense entirely: a reasoning model that achieves 0% ASR under no defense via execution refusal inverts to 100% ASR under Memory Sandbox, because removing explicit recall forces the model onto the RAG pathway where its refusal mechanism does not activate. Memory Sandbox imposes zero utility cost in the absence of attack (BTCR = 100% across all conditions). These results provide the first systematic characterization of why each defense class fails against persistent memory attacks, enabling informed defense investment decisions.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MemLineage: Lineage-Guided Enforcement for LLM Agent Memory

    cs.CR 2026-05 conditional novelty 6.0

    MemLineage enforces untrusted-path persistence in LLM agent memory through Merkle logs, per-principal signatures, and max-of-strong-edges lineage propagation, achieving zero ASR on three poisoning workloads with sub-m...

Reference graph

Works this paper leans on

4 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  3. [3]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  4. [4]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...