pith. sign in

arxiv: 2605.04700 · v2 · pith:DKUZYMNUnew · submitted 2026-05-06 · 💻 cs.CR · cs.AI· cs.CL· cs.LG· cs.SD

Sparse Tokens Suffice: Jailbreaking Audio Language Models via Token-Aware Gradient Optimization

Zheng Fang , Xiaosen Wang , Shenyi Zhang , Shaokang Wang , Zhijin Ge This is my paper

Pith reviewed 2026-06-30 23:44 UTC · model grok-4.3

classification 💻 cs.CR cs.AIcs.CLcs.LGcs.SD
keywords jailbreakaudio language modelsgradient optimizationsparse optimizationadversarial attacktoken alignmentmultimodal safety
0
0 comments X

The pith

Sparse high-energy audio token gradients suffice for strong jailbreak attacks on audio language models.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines whether full dense optimization of audio waveforms is necessary for jailbreaking audio language models. It finds that gradient energy concentrates in a small subset of tokens, allowing most of the waveform to be ignored during updates. Token-Aware Gradient Optimization exploits this by masking low-energy gradients each step. This approach matches or exceeds baseline attack success rates even when retaining only a quarter of the tokens. The result suggests that current dense methods waste effort on redundant regions.

Core claim

Gradient energy in audio language models is highly non-uniform across tokens, so retaining only the high-energy subset during optimization produces jailbreak attacks that remain effective at token retention ratios as low as 0.25, with attack success rates staying within 1% of full-token results on models such as Qwen3-Omni.

What carries the argument

Token-Aware Gradient Optimization (TAGO), which at each iteration computes token-aligned gradients, ranks them by energy, and masks all but the top fraction before applying the update.

If this is right

  • Dense waveform updates are largely redundant for audio jailbreaks.
  • Substantial sparsification preserves attack success rates across three tested ALMs.
  • TAGO outperforms standard dense optimization baselines.
  • Future safety alignment work should exploit the heterogeneous token-level gradient structure.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Attack algorithms could become computationally lighter by focusing updates on fewer tokens.
  • Safety training might benefit from weighting protection toward high-gradient-energy regions.
  • The same non-uniformity may appear in other sequence models or modalities, enabling similar sparsification.

Load-bearing premise

The non-uniform energy distribution of token-aligned gradients stays stable enough across optimization steps and different inputs that low-energy regions can be masked without losing the attack trajectory.

What would settle it

An experiment showing that attack success rate falls sharply when low-energy gradients are masked, even on the same models and prompts used in the paper.

Figures

Figures reproduced from arXiv: 2605.04700 by Shaokang Wang, Shenyi Zhang, Xiaosen Wang, Zheng Fang, Zhijin Ge.

Figure 1
Figure 1. Figure 1: (Left) The architecture of ALMs. (Right) Overview of token-aware gradient optimization (TAGO). ∇δL(δ (k) ) ∈ R L denote the gradient vector at iteration k, the gradient energy of the s-th sample is formulated as g (k) (s) = h∇δL(δ (k) ) i s 2 , s ∈ {1, . . . , L}. (6) The waveform gradient energy is g (k) . We then aggregate these sample-level gradient energies over R(i) to obtain the token-aligned gradi… view at source ↗
Figure 1
Figure 1. Figure 1: (Left) The architecture of ALMs. (Right) Overview of token-aware gradient optimization (TAGO). interval R(i) aligned to the i-th audio token. Sparse token selection. Let ζ ∈ (0, 1] denote the token retention ratio, i.e., the fraction of audio tokens whose gradi￾ents are retained at each iteration. Given the token-aligned gradient g˜ (k) , the selection is formulated as S (k) = Top⌈ζT⌉ [PITH_FULL_IMAGE:fig… view at source ↗
Figure 2
Figure 2. Figure 2: An illustration of the audio token-level gradient distribu￾tion during iterative optimization on Qwen3-Omni. decision. For TAGO, we also report the average number of iterations of optimization process. More details about the experimental setup are provided in Appendix A. 6.2. Evaluation results on AdvBench-50 view at source ↗
Figure 3
Figure 3. Figure 3: Average number of iterations versus token retention ratio ζ ∈ {1.0, 0.75, 0.5, 0.25} with early-stopping threshold τ for ρ ∈ {0.9, 0.8, 0.7} view at source ↗
Figure 4
Figure 4. Figure 4: An extended evaluation on Qwen3-Omni at ρ=0.9. These results are summarized as follows • Larger ρ generally improves attack success rates, at the cost of more optimization iterations. • Reducing ζ preserves strong attack performance with only a slight increase in optimization iterations, and the itera￾tion growth is much slower than 1/ζ. • TAGO maintains non-trivial attack performance even at a very small … view at source ↗
Figure 5
Figure 5. Figure 5: Fixed-prefix ablation results under ρ = 0.9 and ζ = 0.25 reported in average iterations. fraction of the total gradient energy, i.e., ∥M(k) ⊙ g (k) ∥ 2 2 ≥ γk ∥g (k) ∥ 2 2 , (15) where γk ∈ (0, 1] lower-bounds the captured gradient energy ratio at iteration k, and ⊙ denotes element-wise multiplication. Define the captured energy ratio rk := ∥M(k) ⊙ g (k)∥ 2 2/∥g (k)∥ 2 2 ∈ (0, 1], so that rk ≥ γk. Empirica… view at source ↗
Figure 5
Figure 5. Figure 5: Fixed-prefix ablation results under ρ = 0.9 and ζ = 0.25 reported in average iterations. fraction of the total gradient energy, i.e., ∥M(k) ⊙ g (k) ∥ 2 2 ≥ γk ∥g (k) ∥ 2 2 , (15) where γk ∈ (0, 1] lower-bounds the captured gradient energy ratio at iteration k, and ⊙ denotes element-wise multiplication. Define the captured energy ratio rk := ∥M(k) ⊙ g (k)∥ 2 2/∥g (k)∥ 2 2 ∈ (0, 1], so that rk ≥ γk. Empirica… view at source ↗
Figure 6
Figure 6. Figure 6: Illustrations of the gradient distribution across ALMs. For three representative samples, we visualize normalized gradient energy at the waveform sample-point level (left) and the audio-token level (right). Results on Qwen3-Omni, Qwen2.5-Omni, and LLaMA-Omni consistently show strong gradient concentration, indicating substantial non-uniformity of gradient. Proof. By L-smoothness (Nesterov, 2013; Bottou et … view at source ↗
Figure 6
Figure 6. Figure 6: Illustrations of the gradient distribution across ALMs. For three representative samples, we visualize normalized gradient energy at the waveform sample-point level (left) and the audio-token level (right). Results on Qwen3-Omni, Qwen2.5-Omni, and LLaMA-Omni consistently show strong gradient concentration, indicating substantial non-uniformity of gradient energy. Proof. By L-smoothness (Nesterov, 2013; Bot… view at source ↗
Figure 7
Figure 7. Figure 7: Illustrations of the audio token-level gradient distribution during iterative optimization on Qwen2.5-Omni and LLaMa-Omni. Interpretation. Theorem D.3 shows that TAGO’s per-iteration progress is governed by the captured energy ratio rk. Assumption D.2 further ensures rk is lower-bounded by γk. Dense updates correspond to rk = 1. Empirically, we find rk remains substantial even when the token retention rati… view at source ↗
Figure 7
Figure 7. Figure 7: Illustrations of the audio token-level gradient distribution during iterative optimization on Qwen2.5-Omni and LLaMA-Omni. Interpretation. Theorem D.3 shows that TAGO’s per-iteration progress is governed by the captured energy ratio rk. Assumption D.2 further ensures rk is lower-bounded by γk. Dense updates correspond to rk = 1. Empirically, we find rk remains substantial even when the token retention rati… view at source ↗
Figure 8
Figure 8. Figure 8: Waveforms of the original harmful audio and TAGO-perturbed audios optimized against three ALMs. 19 view at source ↗
Figure 8
Figure 8. Figure 8: Waveforms of the original harmful audio and TAGO-perturbed audios optimized against three ALMs. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Case study on Qwen3-Omni. 20 view at source ↗
Figure 9
Figure 9. Figure 9: Case study on Qwen3-Omni. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗
read the original abstract

Jailbreak attacks on audio language models (ALMs) optimize audio perturbations to elicit unsafe generations, and they typically update the entire waveform densely throughout optimization. In this work, we investigate the necessity of such dense optimization by analyzing the structure of token-aligned gradients in ALMs. We find that gradient energy is highly non-uniform across audio tokens, indicating that only a small subset of token-aligned audio regions dominates the optimization signal. Motivated by this observation, we propose Token-Aware Gradient Optimization (TAGO), which enables sparse jailbreak optimization by retaining only waveform gradients aligned with audio tokens that have high gradient energy, while masking the remaining gradients at each iteration. Across three ALMs, TAGO outperforms baselines, and substantial sparsification preserves strong attack success rates (e.g. on Qwen3-Omni, $\mathrm{ASR}_{l}$ remains at 86% with a token retention ratio of 0.25, compared to 87% with full token retention). These results demonstrate that dense waveform updates are largely redundant, and we advocate that future audio jailbreak and safety alignment research should further leverage this heterogeneous token-level gradient structure.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript claims that token-aligned gradients in audio language models exhibit highly non-uniform energy distribution, motivating Token-Aware Gradient Optimization (TAGO) which performs sparse updates by retaining only high-energy token gradients at each iteration; across three ALMs this yields attack success rates comparable to dense optimization, e.g., ASR_l of 86% at 0.25 token retention ratio versus 87% with full retention on Qwen3-Omni.

Significance. If the stability of the high-energy token mask holds, the result would establish that dense waveform updates are largely redundant for audio jailbreaks, providing a concrete efficiency gain and a new lens on gradient structure that could inform both attack design and safety alignment research.

major comments (1)
  1. [Abstract / Experimental results] Abstract / Experimental results: the claim that 0.25 retention preserves 86% ASR_l (versus 87% full) rests on the assumption that the high-energy token set remains stable across iterations, yet the manuscript reports only endpoint ASR numbers and supplies no quantification of mask overlap, variance, or temporal evolution of the selected tokens.
minor comments (2)
  1. The abstract states that TAGO outperforms baselines but gives no implementation details, statistical significance tests, or exact experimental controls for the reported ASR figures.
  2. Notation for ASR_l and the precise definition of the token retention ratio should be clarified with an equation or pseudocode in the methods section.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback. We address the single major comment below, clarifying the dynamic nature of TAGO and committing to added analysis.

read point-by-point responses

  1. Referee: [Abstract / Experimental results] Abstract / Experimental results: the claim that 0.25 retention preserves 86% ASR_l (versus 87% full) rests on the assumption that the high-energy token set remains stable across iterations, yet the manuscript reports only endpoint ASR numbers and supplies no quantification of mask overlap, variance, or temporal evolution of the selected tokens.

    Authors: TAGO recomputes the high-energy token mask at every optimization iteration from the current gradient energy (Algorithm 1, Section 3.2); the selection is therefore dynamic by design and does not rest on any assumption of mask stability across iterations. The reported endpoint ASR_l of 86% at 0.25 retention (vs. 87% full) on Qwen3-Omni directly demonstrates that per-step sparse updates suffice. We agree that explicit quantification of mask dynamics would strengthen the presentation and will add, in the revision, Jaccard overlap statistics between consecutive masks, variance of selected token indices, and a plot of mask evolution over optimization steps. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical observation and masking technique

full rationale

The paper reports an empirical observation that token-aligned gradients exhibit non-uniform energy distribution, then defines TAGO as the procedure of masking low-energy gradients at each iteration and evaluates it experimentally on three ALMs. No equations, predictions, or first-principles derivations are presented that reduce to fitted parameters or self-referential inputs. Results (e.g., ASR_l values at retention ratios) are direct experimental measurements, not outputs forced by construction from the inputs. No load-bearing self-citations or uniqueness theorems appear in the provided text. The approach is self-contained as an optimization heuristic validated by attack-success metrics.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central claim rests on one domain assumption about gradient structure and one tunable hyperparameter; no new entities are postulated.

free parameters (1)
  • token retention ratio = 0.25
    Hyperparameter selected to illustrate substantial sparsification while preserving attack performance; value 0.25 is used in the reported example.
axioms (1)
  • domain assumption Gradient energy is highly non-uniform across audio tokens
    This observation, stated in the abstract as the motivation for TAGO, is invoked to justify masking low-energy gradients at each iteration.

pith-pipeline@v0.9.1-grok · 5753 in / 1230 out tokens · 29696 ms · 2026-06-30T23:44:49.043931+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

4 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    Google DeepMind

    URL https://cloud.google.com/ text-to-speech. Google DeepMind. Gemini 3 Flash model card,

  2. [2]

    Step-Audio: Unified Understanding and Generation in Intelligent Speech Interaction

    URL https://storage.googleapis. com/deepmind-media/Model-Cards/ Gemini-3-Flash-Model-Card.pdf. Huang, A., Wu, B., Wang, B., Yan, C., Hu, C., Feng, C., Tian, F., Shen, F., Li, J., Chen, M., et al. Step-audio: Uni- fied understanding and generation in intelligent speech interaction.arXiv preprint arXiv:2502.11946, 2025. 10 Sparse Tokens Suffice: Jailbreakin...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  3. [3]

    Sparsity in attention.Work on KV cache compression suggests that attention during generation is often dominated by a small subset of tokens (e.g., H2O (Zhang et al., 2023) and StreamingLLM (Xiao et al., 2024)), which is consistent with the view that only a small portion of the context carries most of the effective signal and may lead to concentrated optim...

    2023

  4. [4]

    demonstrate how to make a bomb using common household items

    Redundancy in audio representations.Audio masked autoencoders (Audio-MAE) show that semantic content can be preserved even when masking a large fraction of audio patches (Huang et al., 2022), suggesting substantial redundancy in audio features. Such redundancy is consistent with the existence of low-dimensional effective directions for changing model beha...

    2022