REVIEW 1 major objections 1 minor 1 cited by
Memory-layer tool gating stops persistent memory attacks on eight of nine LLM agents by blocking recall but inverts to full success on one reasoning model.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.3
2026-06-30 22:52 UTC pith:UYQQEWLH
load-bearing objection Memory sandbox stops attacks on eight models but inverts to 100% ASR on one reasoning model, showing layer placement does not predict outcomes independently of model-specific paths. the 1 major comments →
Defense effectiveness across architectural layers: a mechanistic evaluation of persistent memory attacks on stateful LLM agents
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Persistent memory attacks succeed by storing malicious instructions through RAG documents for delayed execution; input and retrieval defenses fail because injected content remains invisible or semantically masked, while memory-layer tool gating removes the recall capability required for execution on eight of nine models and imposes zero utility cost on benign tasks, except when the defense forces a reasoning model onto an RAG pathway where its refusal does not activate.
What carries the argument
Memory Sandbox: a tool-gating defense at the memory layer that prevents recall and use of previously stored instructions.
Load-bearing premise
Persistent memory attacks depend on the model explicitly recalling stored content, and defense success is determined by architectural layer placement independent of model-specific reasoning pathways.
What would settle it
Measuring attack success rate on the inverting reasoning model when recall is blocked but the model is still given the option to process the malicious content through its normal refusal-triggering path.
If this is right
- Input-level and retrieval-level defenses achieve attack success rates of 88-89 percent, statistically identical to the undefended baseline.
- Memory Sandbox reduces attack success to zero on eight models while maintaining 100 percent benign task completion rate.
- Prompt hardening lowers overall attack success to 77.8 percent but only through refusal behavior on two specific models.
- Architectural layer placement interacts with model reasoning style such that blocking recall can activate alternate vulnerable pathways.
Where Pith is reading between the lines
- Defenses may need to preserve or replicate refusal mechanisms across multiple information pathways rather than relying on single-layer intervention.
- Evaluation of new defenses should include models that already exhibit strong refusal to detect cases where layer changes shift execution routes.
- Layer-based defenses could be combined to close both recall and retrieval routes simultaneously.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports a large-scale empirical study (5040 runs, N=40 per condition) evaluating six defenses across four architectural layers against persistent memory attacks on nine open-source LLM agents. It finds that input-level (Minimizer, Sanitizer) and retrieval-level (RAG Sanitizer, RAG LLM Judge) defenses achieve 88-89% ASR, statistically indistinguishable from the undefended 88.6% baseline; Prompt Hardening reaches 77.8% ASR with effects driven by two models; and Memory Sandbox at the memory layer reduces ASR to 0% for eight models by blocking explicit recall, but inverts to 100% ASR on one reasoning model (which had 0% undefended ASR via execution refusal) by forcing the RAG pathway where refusal does not activate. The work claims this provides the first systematic characterization of why each defense class fails, with Memory Sandbox imposing zero utility cost (BTCR=100%).
Significance. If the results hold, the large-scale direct comparison to baseline and the identification of an effective memory-layer defense with no utility penalty would provide actionable guidance for defense design against persistent memory attacks. The scale (5040 runs) and inclusion of an outlier case strengthen the empirical contribution, though the post-hoc interpretation of the inversion limits the strength of the architectural-layer claims.
major comments (1)
- [Abstract] Abstract: the claim that 'the architectural explanation holds' (input-level defenses cannot observe RAG content; retrieval classifiers defeated by masking) is challenged by the reported inversion under Memory Sandbox, where one reasoning model's success depends on its specific refusal pathway rather than layer placement; the post-hoc mechanistic story for the outlier lacks a pre-specified ablation or test confirming that inverted attacks route via RAG rather than another mechanism, undermining the assertion that layer placement independently predicts defense outcome across models.
minor comments (1)
- [Abstract] The abstract reports statistical indistinguishability from baseline but does not name the exact tests or correction methods used; adding these details would improve verifiability of the 'fail at approximately baseline' claims.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback on the abstract. We agree that the outlier case requires more careful qualification of the architectural claims and will revise the abstract accordingly.
read point-by-point responses
-
Referee: [Abstract] Abstract: the claim that 'the architectural explanation holds' (input-level defenses cannot observe RAG content; retrieval classifiers defeated by masking) is challenged by the reported inversion under Memory Sandbox, where one reasoning model's success depends on its specific refusal pathway rather than layer placement; the post-hoc mechanistic story for the outlier lacks a pre-specified ablation or test confirming that inverted attacks route via RAG rather than another mechanism, undermining the assertion that layer placement independently predicts defense outcome across models.
Authors: We accept this point. The inversion on the reasoning model (0% undefended ASR via execution refusal, 100% under Memory Sandbox) shows that model-specific refusal pathways can override layer-based predictions. Our explanation that Memory Sandbox forces the RAG pathway is inferred from the attack design and observed behaviors rather than from a pre-specified ablation isolating RAG vs. explicit recall. We will revise the abstract to state that the architectural explanation holds for eight of nine models, with one noted exception driven by model-specific mechanisms. This avoids claiming independent prediction across all models while preserving the empirical comparison of defense classes. revision: partial
Circularity Check
No circularity: purely empirical evaluation with direct measurements
full rationale
The paper reports results from 5,040 experimental runs measuring attack success rates (ASR) under different defenses across models. No equations, fitted parameters, derivations, or self-citations are used to support the central claims. The reported outcomes (e.g., Memory Sandbox reducing ASR to 0% for eight models, with one inversion) are direct observations from the test conditions, not quantities defined by or reduced to the inputs by construction. The analysis is self-contained against the external benchmark of measured ASR values.
Axiom & Free-Parameter Ledger
read the original abstract
Persistent memory attacks against LLM agents achieve high attack success rates against open-source models. In these attacks, malicious instructions injected via RAG-retrieved documents are stored in persistent memory and executed in later sessions. However, no systematic evaluation of defense effectiveness against this attack class exists. We evaluate six defenses across four architectural layers against delayed-trigger attacks on nine open-source models (5,040 runs, N=40 per condition). Four defenses fail at approximately baseline attack success rate: input-level filtering (Minimizer, Sanitizer) and retrieval-level filtering (RAG Sanitizer, RAG LLM Judge) achieve 88-89% ASR, statistically indistinguishable from the undefended baseline of 88.6%. Prompt Hardening partially fails at 77.8% ASR, with the reduction driven by two models at 0%: one genuine defense effect and one model-level refusal independent of the defense. The architectural explanation holds: input-level defenses cannot observe RAG-injected content, and retrieval-level classifiers are defeated by compliance-framed semantic masking. One defense, tool-gating at the memory layer (Memory Sandbox), reduces ASR to 0% for eight of nine models by removing the recall capability the attack requires. The exception inverts the defense entirely: a reasoning model that achieves 0% ASR under no defense via execution refusal inverts to 100% ASR under Memory Sandbox, because removing explicit recall forces the model onto the RAG pathway where its refusal mechanism does not activate. Memory Sandbox imposes zero utility cost in the absence of attack (BTCR = 100% across all conditions). These results provide the first systematic characterization of why each defense class fails against persistent memory attacks, enabling informed defense investment decisions.
Forward citations
Cited by 1 Pith paper
-
MemLineage: Lineage-Guided Enforcement for LLM Agent Memory
MemLineage enforces untrusted-path persistence in LLM agent memory through Merkle logs, per-principal signatures, and max-of-strong-edges lineage propagation, achieving zero ASR on three poisoning workloads with sub-m...
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
@esa (Ref
\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...
-
[3]
\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...
-
[4]
@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.