Pith. sign in

REVIEW 2 major objections 1 minor 1 cited by

Overlapping concepts in language models create a cylindrical structure that accounts for steering fluctuations.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-07-01 00:07 UTC pith:44MJ5GBC

load-bearing objection CRH offers a geometric account of steering instability by relaxing LRH orthogonality, but the load-bearing claim that difference vectors cleanly yield an identifiable axis and plane while sectors remain unidentifiable is undercut by likely concept entanglement. the 2 major comments →

arxiv 2605.01844 v2 pith:44MJ5GBC submitted 2026-05-03 cs.CL

The Cylindrical Representation Hypothesis for Language Model Steering

Lang Gao , Jinghui Zhang , Wei Liu , Fengxian Ji , Chenxi Wang , Zirui Song , Akash Ghosh , Youssef Mohamed
show 2 more authors
Preslav Nakov Xiuying Chen
This is my paper
classification cs.CL
keywords cylindrical representation hypothesislanguage model steeringlinear representation hypothesisconcept representationssteering sensitivityactivation uncertaintyoverlapping concepts
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes that relaxing the orthogonality requirement of the Linear Representation Hypothesis while keeping linear representations leads to a sample-specific cylindrical structure for concepts. This structure consists of a central axis that drives the difference between concept presence and absence, surrounded by a normal plane that determines how sensitive steering is to activation. Within that plane, only certain sectors strongly activate the concept, while others suppress it, creating uncertainty because the sector cannot be reliably identified from difference vectors. A reader would care because this explains why steering large language models often produces unpredictable results even with good direction vectors. The experiments confirm this cylindrical form exists in practice and helps interpret real steering behavior.

Core claim

By relaxing LRH's orthogonality assumption while preserving linear representations, overlapping concept contributions naturally yield a sample-specific axis-orthogonal structure formalized as the Cylindrical Representation Hypothesis (CRH). In CRH, a central axis captures the main difference between concept absence and presence and drives concept generation. A surrounding normal plane controls steering sensitivity by determining how easily the axis can activate the target concept. Within this plane, only specific sensitive sectors strongly facilitate concept activation, while other sectors can suppress or delay it. The surrounding normal plane can be reliably identified from difference vecto

What carries the argument

The Cylindrical Representation Hypothesis (CRH), a model of concept representation with a central axis for presence difference and a normal plane containing sensitive sectors that control activation ease.

Load-bearing premise

The surrounding normal plane can be reliably identified from difference vectors while the sensitive sector cannot, thereby introducing intrinsic uncertainty at the sector level.

What would settle it

Finding that steering outcomes remain stable and predictable even without knowing the sensitive sector, or failing to observe the cylindrical structure in activation difference vectors across samples.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Steering using the identified axis direction will still vary in outcome depending on the sample's location in the normal plane.
  • The reliable identification of the normal plane allows consistent axis finding, but sector uncertainty causes observed fluctuations.
  • Concept activation can be facilitated or delayed based on the sector position relative to the sensitive one.
  • Linear representations are preserved, but overlaps lead to this cylindrical rather than purely orthogonal form.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Steering methods might be improved by attempting to estimate or average over possible sectors in the plane.
  • This cylindrical view could apply to other control techniques beyond steering in language models.
  • Visualizing the normal plane for multiple samples might reveal patterns in sector locations.
  • Extensions could test whether training affects the size or structure of the sensitive sectors.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper claims that relaxing the orthogonality assumption of the Linear Representation Hypothesis (LRH) while preserving linear representations leads to overlapping concept contributions that naturally produce a sample-specific cylindrical structure. This is formalized as the Cylindrical Representation Hypothesis (CRH), in which a central axis (recovered from difference vectors) captures the main difference between concept absence and presence and drives generation, a surrounding normal plane controls steering sensitivity, and only specific sensitive sectors within that plane facilitate activation while others suppress it. The normal plane is asserted to be reliably identifiable from difference vectors, but the sensitive sector is not, introducing intrinsic uncertainty that explains steering fluctuations. Experiments are reported to verify the cylindrical structure and demonstrate CRH's utility for interpreting real steering behavior, with code provided.

Significance. If the cylindrical geometry and the claimed identifiability separation hold, CRH supplies a concrete geometric account of steering instability that remains within the linear-representation paradigm and could guide more stable steering methods. The inclusion of experiments together with a public code repository is a strength that supports empirical scrutiny and reproducibility.

major comments (2)
  1. [Abstract / CRH formalization] Abstract and the formalization of CRH: the load-bearing assertion that 'the surrounding normal plane can be reliably identified from difference vectors' while the sensitive sector cannot must be justified against the possibility that difference vectors entangle multiple overlapping concepts. The hypothesis itself invokes overlapping contributions, yet no explicit argument or isolation mechanism is supplied showing why the recovered axis and plane remain concept-pure.
  2. [Experiments] Experiments section: verification of the cylindrical structure should include controls or ablations that test robustness of axis and plane recovery when difference vectors are constructed under controlled concept overlap, directly addressing whether the claimed reliable identification of the normal plane survives the regime the hypothesis assumes.
minor comments (1)
  1. The GitHub link is mentioned only in the abstract; a stable citation or footnote in the main text would improve traceability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive review. The comments highlight important points regarding the justification of CRH's identifiability claims and the need for additional experimental controls. We address each below and commit to revisions that strengthen the manuscript without altering its core claims.

read point-by-point responses

  1. Referee: [Abstract / CRH formalization] Abstract and the formalization of CRH: the load-bearing assertion that 'the surrounding normal plane can be reliably identified from difference vectors' while the sensitive sector cannot must be justified against the possibility that difference vectors entangle multiple overlapping concepts. The hypothesis itself invokes overlapping contributions, yet no explicit argument or isolation mechanism is supplied showing why the recovered axis and plane remain concept-pure.

    Authors: We agree that an explicit isolation argument is needed. The CRH formalization defines the axis as the principal direction of difference vectors (concept-present minus concept-absent), which isolates the dominant linear contribution even when secondary concepts overlap in the representation space. The normal plane is then the orthogonal complement, recoverable via the null space of the axis. However, the manuscript does not provide a dedicated lemma or decomposition showing why overlap does not contaminate the plane identification. We will add a short mathematical subsection deriving this from the linear superposition model, demonstrating that the axis remains the leading eigenvector and the plane is identifiable as long as the target concept has non-zero projection on the difference vector. revision: yes

  2. Referee: [Experiments] Experiments section: verification of the cylindrical structure should include controls or ablations that test robustness of axis and plane recovery when difference vectors are constructed under controlled concept overlap, directly addressing whether the claimed reliable identification of the normal plane survives the regime the hypothesis assumes.

    Authors: This is a valid request for stronger validation. Our existing experiments demonstrate the cylindrical geometry on natural activations and steering tasks, but they do not include synthetic ablations with injected concept overlap. We will add a controlled ablation subsection that generates difference vectors under varying overlap levels (via linear combinations of known directions) and reports metrics on axis recovery accuracy and plane stability. This directly tests the identifiability claim under the overlapping regime assumed by CRH. revision: yes

Circularity Check

0 steps flagged

No significant circularity; CRH is a proposed formalization from relaxing LRH, not a reduction to inputs.

full rationale

The paper's central move is to relax the orthogonality assumption of the existing Linear Representation Hypothesis (LRH) while keeping linear representations, then define the Cylindrical Representation Hypothesis (CRH) as the resulting structure. No equations, parameters, or predictions are shown to be fitted to data and then renamed as outputs. No self-citations are invoked as load-bearing uniqueness theorems. The derivation is presented as a theoretical relaxation that yields a new descriptive structure, with experiments offered as verification rather than as the source of the claim. This is self-contained against external benchmarks and receives the default non-finding.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no explicit free parameters, background axioms, or independently evidenced invented entities; the cylindrical structure itself is postulated to explain steering behavior.

pith-pipeline@v0.9.1-grok · 5791 in / 1119 out tokens · 30825 ms · 2026-07-01T00:07:18.297839+00:00 · methodology

0 comments
read the original abstract

Steering is a widely used technique for controlling large language models, yet its effects are often unstable and hard to predict. Existing theoretical accounts are largely based on the Linear Representation Hypothesis (LRH). While LRH assumes that concepts can be orthogonalized for lossless control, this idealized mapping fails in real representations and cannot account for the observed unpredictability of steering. By relaxing LRH's orthogonality assumption while preserving linear representations, we show that overlapping concept contributions naturally yield a sample-specific axis-orthogonal structure. We formalize this as the Cylindrical Representation Hypothesis (CRH). In CRH, a central axis captures the main difference between concept absence and presence and drives concept generation. A surrounding normal plane controls steering sensitivity by determining how easily the axis can activate the target concept. Within this plane, only specific sensitive sectors strongly facilitate concept activation, while other sectors can suppress or delay it. While the surrounding normal plane can be reliably identified from difference vectors, the sensitive sector cannot, introducing intrinsic uncertainty at the sector level. This uncertainty provides a principled explanation for why steering outcomes often fluctuate even when using well-aligned directions. Our experiments verify the existence of the cylindrical structure and demonstrate that CRH provides a valid and practical way to interpret model steering behavior in real settings: https://github.com/mbzuai-nlp/CRH.

Figures

Figures reproduced from arXiv: 2605.01844 by Akash Ghosh, Chenxi Wang, Fengxian Ji, Jinghui Zhang, Lang Gao, Preslav Nakov, Wei Liu, Xiuying Chen, Youssef Mohamed, Zirui Song.

Figure 1
Figure 1. Figure 1: Comparison of LRH and CRH. LRH assumes a single global concept direction, while CRH reveals a sample-specific cylindrical structure with a central axis and an orthogonal normal plane. Steering outcomes depend on the sector within the normal plane, exposing the sample-specific nature of steerability. 1. Introduction As large language models (LLMs) become more capable, researchers have become increasingly in… view at source ↗
Figure 2
Figure 2. Figure 2: A counterexample of independent control. Three in￾dependent concepts defined in a three-dimensional latent space cannot remain orthogonal when represented in two dimensions. As a result, intervening on one concept inevitably affects others. Steering modifies model outputs at inference time by adding a concept-related vector to internal representations, typically constructed from differences between positiv… view at source ↗
Figure 3
Figure 3. Figure 3: Geometric structure induced by CRH. (a) Each concept vector v (i) decomposes into components parallel and orthogonal to the difference vector vd, and vd is a weighted sum of concept vectors that defines the central axis. (b) The orthogonal components balance and form a sample-specific normal plane Pd. (c) A steering vector splits into an axis-aligned and a normal-plane component, whose phase determines whe… view at source ↗
Figure 4
Figure 4. Figure 4: Non-predictability of steering effectiveness on normal￾plane phase, where a single difference-vector direction can cor￾respond to multiple possible concept compositions and induce different sensitive sectors. Because of this containment, the two projections satisfy a nesting relation, which ensures that the observable projec￾tion captures all variations of the latent one. By examining their gradients, one … view at source ↗
Figure 5
Figure 5. Figure 5: Probed cylindrical structure of CRH for a fixed sample. (a) We show the loss distribution over the entire cylindrical structure. (b) We plot loss trajectories along the axis for the phases with the minimum and maximum average loss. (c) We present normalized loss distributions over the normal plane at selected steering steps, showing stable sector patterns across steps. For each plane, we show the outputs c… view at source ↗
Figure 7
Figure 7. Figure 7: Verification of CRH determinability. (a) The maxi￾mum correlation exhibits a single peak as the total exponent in￾creases, with the minimum p-value attained near the peak. (b) The difference-vector similarity does not correlate with steerability similarity for a fixed concept (Gemma-2B-IT, layer 9). We define the steerability score as Stc(v) = a ∥v∥ , (17) which represents the increase in the success rate … view at source ↗
Figure 8
Figure 8. Figure 8: Results of different vector construction methods on Gemma-2B-IT. The shaded area indicates the 95% confidence interval across concepts. 5 10 15 20 25 Step 0% 5% 10% Rate Rate of Outputs with Target Concept DiffMean PCA MC Probe 5 10 15 20 25 Step 0% 50% 100% Rate Rate of Corrupted Outputs DiffMean PCA MC Probe (a) Target activation and corruption rates over steps (Layer 16). 5 10 15 20 25 Step 0% 5% 10% 15… view at source ↗
Figure 9
Figure 9. Figure 9: Results of different vector construction methods on Llama2-7B-Chat. The shaded area indicates the 95% confidence interval across concepts. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Probed cylindrical structure of CRH for a fixed sample. (a) The loss distribution over the entire cylindrical structure. (b) We plot loss trajectories along the axis for the phases with the minimum and maximum average loss. (c) We present normalized loss distributions over the normal plane at selected steering steps, showing stable sector patterns across steps. For each plane, we show outputs correspondin… view at source ↗
Figure 11
Figure 11. Figure 11: Probed cylindrical structure of CRH for a fixed sample. (a) The loss distribution over the entire cylindrical structure. (b) We plot loss trajectories along the axis for the phases with the minimum and maximum average loss. (c) We present normalized loss distributions over the normal plane at selected steering steps, showing stable sector patterns across steps. For each plane, we show outputs correspondin… view at source ↗
Figure 12
Figure 12. Figure 12: Probed cylindrical structure of CRH for a fixed sample. (a) The loss distribution over the entire cylindrical structure. (b) We plot loss trajectories along the axis for the phases with the minimum and maximum average loss. (c) We present normalized loss distributions over the normal plane at selected steering steps, showing stable sector patterns across steps. For each plane, we show outputs correspondin… view at source ↗
Figure 13
Figure 13. Figure 13: Effect of penalizing the normal-plane component on steering outcomes: (a) target concept activation and (b) output corruption, illustrating the trade-off predicted by CRH. Results for steering all prompt tokens. (a) Gemma-2B-IT Layer 9 (b) Llama-7B-Chat Layer 16 Layer 24 Layer 13 Last Prompt Tks [PITH_FULL_IMAGE:figures/full_fig_p023_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Effect of penalizing the normal-plane component on steering outcomes: (a) target concept activation and (b) output corruption, illustrating the trade-off predicted by CRH. Results for steering last prompt token only. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Effect of penalizing the normal-plane component on steering outcomes: (a) target concept activation and (b) output corruption, illustrating the trade-off predicted by CRH. Results for steering all output tokens. (a) Gemma-2B-IT Layer 9 (b) Llama-7B-Chat Layer 16 Layer 24 Layer 13 ALL Tokens [PITH_FULL_IMAGE:figures/full_fig_p024_15.png] view at source ↗
Figure 16
Figure 16. Figure 16 [PITH_FULL_IMAGE:figures/full_fig_p024_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Linear predictability of DiffMean when steering all prompt tokens on (a) Gemma-2B-IT and (b) Llama2-7B-Chat. PCA (a) Gemma-2B-IT Layer 9 (b) Llama-7B-Chat Probe Layer 13 Layer 16 Layer 24 (a) Gemma-2B-IT Layer 9 (b) Llama-7B-Chat Layer 13 Layer 16 Layer 24 [PITH_FULL_IMAGE:figures/full_fig_p025_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Linear predictability of PCA when steering all prompt tokens on (a) Gemma-2B-IT and (b) Llama2-7B-Chat. (a) Gemma-2B-IT Layer 9 (b) Llama-7B-Chat Main MC Layer 13 Layer 16 Layer 24 (a) Gemma-2B-IT Layer 9 (b) Llama-7B-Chat Layer 13 Layer 16 Layer 24 [PITH_FULL_IMAGE:figures/full_fig_p025_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Linear Predictability of Mean-Centering when steering all prompt tokens on (a) Gemma-2B-IT and (b) Llama2-7B-Chat. PCA (a) Gemma-2B-IT Layer 9 (b) Llama-7B-Chat Probe Layer 13 Layer 16 Layer 24 (a) Gemma-2B-IT Layer 9 (b) Llama-7B-Chat Layer 13 Layer 16 Layer 24 [PITH_FULL_IMAGE:figures/full_fig_p025_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Linear Predictability of Probe when steering all prompt tokens on (a) Gemma-2B-IT and (b) Llama2-7B-Chat. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Non-linear predictability of DiffMean when steering all prompt tokens on (a) Gemma-2B-IT and (b) Llama2-7B-Chat. PCA Probe (a) Gemma-2B-IT Layer 9 Layer 13 (b) Llama-7B-Chat Layer 16 Layer 24 (a) Gemma-2B-IT Layer 9 Layer 13 (b) Llama-7B-Chat Layer 16 Layer 24 [PITH_FULL_IMAGE:figures/full_fig_p026_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Non-linear predictability of PCA when steering all prompt tokens on (a) Gemma-2B-IT and (b) Llama2-7B-Chat. (a) Gemma-2B-IT Layer 9 Layer 13 (b) Llama-7B-Chat Layer 16 Layer 24 Main MC (a) Gemma-2B-IT Layer 9 Layer 13 (b) Llama-7B-Chat Layer 16 Layer 24 [PITH_FULL_IMAGE:figures/full_fig_p026_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Non-linear Predictability of Mean-Centering when steering all prompt tokens on (a) Gemma-2B-IT and (b) Llama2-7B-Chat. PCA Probe (a) Gemma-2B-IT Layer 9 Layer 13 (b) Llama-7B-Chat Layer 16 Layer 24 (a) Gemma-2B-IT Layer 9 Layer 13 (b) Llama-7B-Chat Layer 16 Layer 24 [PITH_FULL_IMAGE:figures/full_fig_p026_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Non-linear Predictability of Probe when steering all prompt tokens on (a) Gemma-2B-IT and (b) Llama2-7B-Chat. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Validation of the three implications on Qwen2.5-14B-Instruct. The larger model shows the same patterns as Gemma-2B-IT and LLaMA2-7B-Chat. K.3. Additional Verification Motivation. In the main text, we use Gemma-2B-IT and LLaMA2-7B-Chat for evaluation. To further verify the scope of CRH, we run the same evaluation as in Section 6 on a larger model. Setup. We use Qwen2.5-14B-Instruct (Yang et al., 2025) as t… view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Decodable but Not Corrected by Fixed Residual-Stream Linear Steering: Evidence from Medical LLM Failure Regimes

    cs.AI 2026-05 unverdicted novelty 5.0

    Overthinking in medical QA is linearly decodable at 71.6% accuracy yet fixed residual-stream steering yields no correction across 29 configurations, while enabling selective abstention with AUROC 0.610.

Reference graph

Works this paper leans on

20 extracted references · 15 canonical work pages · cited by 1 Pith paper · 7 internal anchors

  1. [1]

    arXiv preprint arXiv:2511.18284 , year=

    Bas, T. and Novak, K. What can we actually steer? A multi-behavior study of activation control.ArXiv preprint, arXiv:2511.18284,

    work page arXiv

  2. [2]

    A., and Krasheninnikov, D

    Braun, J., Eickhoff, C., Krueger, D., Bahrainian, S. A., and Krasheninnikov, D. Understanding (un)reliability of steering vectors in language models. InProceedings of the ICLR 2025 Workshop on Building Trust in Lan- guage Models and Applications, ICLR Workshop ’25, Singapore,

    2025

  3. [3]

    Brumley, J

    Brumley, M., Kwon, J., Krueger, D., Krasheninnikov, D., and Anwar, U. Comparing bottom-up and top-down steering approaches on in-context learning tasks.ArXiv preprint, arXiv:2411.07213,

    work page arXiv

  4. [4]

    arXiv preprint arXiv:2411.02193 , year=

    Curran Associates, Inc. Chalnev, S., Siu, M., and Conmy, A. Improving steering vectors by targeting sparse autoencoder features.ArXiv preprint, arXiv:2411.02193,

    work page arXiv

  5. [5]

    Toy Models of Superposition

    Elhage, N., Hume, T., Olsson, C., et al. Toy models of superposition.ArXiv preprint, arXiv:2209.10652,

    work page internal anchor Pith review Pith/arXiv arXiv

  6. [6]

    Gemma: Open Models Based on Gemini Research and Technology

    Association for Computational Linguistics. Gemma Team, Mesnard, T., Hardin, C., Dadashi, R., et al. Gemma: Open models based on Gemini research and technology.ArXiv preprint, arXiv:2403.08295,

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    The Llama 3 Herd of Models

    Grattafiori, A., Dubey, A., Jauhri, A., et al. The Llama 3 herd of models.ArXiv preprint, arXiv:2407.21783,

    work page internal anchor Pith review Pith/arXiv arXiv

  8. [8]

    Jiang, E

    OpenReview.net. Jiang, E. H., Ou, W., Liu, R., Pang, S., Wan, G., Duan, R., Dong, W., Chang, K.-W., Wang, X., Wu, Y . N., and Li, X. Energy-driven steering: Reducing false refusals in large language models.ArXiv preprint, arXiv:2510.08646,

    work page arXiv

  9. [9]

    Improving activation steering in language models with mean-centring, 2023

    Jorgensen, O., Cope, D., Schoots, N., and Shanahan, M. Improving activation steering in language models with mean-centring.ArXiv preprint, arXiv:2312.03813,

    work page arXiv

  10. [10]

    The origins of representation manifolds in large language models, 2025

    Modell, A., Rubin-Delanchy, P., and Whiteley, N. The ori- gins of representation manifolds in large language models. ArXiv preprint, arXiv:2505.18235,

    work page arXiv

  11. [11]

    Nguyen, T

    OpenReview.net. Nguyen, T. and Leng, Y . Toward a flexible framework for linear representation hypothesis using maximum like- lihood estimation.ArXiv preprint, arXiv:2502.16385,

    work page arXiv

  12. [12]

    F., Marks, L., Barez, F., and Abdullah, A

    Oozeer, N. F., Marks, L., Barez, F., and Abdullah, A. Be- yond linear steering: Unified multi-attribute control for language models. InFindings of the Association for Computational Linguistics: EMNLP 2025, EMNLP Find- ings ’25, pp. 23513–23557, Suzhou, China,

    2025

  13. [13]

    P.; Galley, M.; Caruana, R.; Gao, J

    Association for Computational Linguistics. Singh, C., Inala, J. P., Galley, M., Caruana, R., and Gao, J. Rethinking interpretability in the era of large language models.ArXiv preprint, arXiv:2402.01761,

    work page arXiv

  14. [14]

    Llama 2: Open Foundation and Fine-Tuned Chat Models

    Curran Asso- ciates, Inc. Touvron, H., Martin, L., Stone, K., et al. Llama 2: Open foundation and fine-tuned chat models.ArXiv preprint, arXiv:2307.09288,

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [15]

    Qwen2.5 Technical Report

    PMLR. Yang, A., Yang, B., Zhang, B., Hui, B., et al. Qwen2.5 technical report.ArXiv preprint, arXiv:2412.15115,

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models

    Zhang, H., Zhang, Z., Wang, M., et al. Locate, steer, and improve: A practical survey of actionable mechanistic interpretability in large language models.ArXiv preprint, arXiv:2601.14004,

    work page internal anchor Pith review Pith/arXiv arXiv

  17. [17]

    Zhao, H., Wu, X., Yang, F., Shen, B., Liu, N., and Du, M

    Association for Computational Linguistics. Zhao, H., Wu, X., Yang, F., Shen, B., Liu, N., and Du, M. Denoising concept vectors with sparse autoencoders for improved language model steering. InFindings of the Association for Computational Linguistics: EACL 2026, EACL Findings ’26, pp. 797–808, Rabat, Morocco,

    2026

  18. [18]

    Representation Engineering: A Top-Down Approach to AI Transparency

    Curran Associates, Inc. Zou, A., Phan, L., Chen, S., et al. Representation engineer- ing: A top-down approach to AI transparency.ArXiv preprint, arXiv:2310.01405,

    work page internal anchor Pith review Pith/arXiv arXiv

  19. [19]

    Limitations While CRH offers a new geometric perspective for interpreting steering behavior, several limitations should be noted

    11 The Cylindrical Representation Hypothesis for Language Model Steering A. Limitations While CRH offers a new geometric perspective for interpreting steering behavior, several limitations should be noted. First, CRH is proposed as a conceptual framework rather than a directly verifiable property of model representations. It assumes that sample-level diff...

    2023

  20. [20]

    C/C++ syntax

    Notably, the LLM judge achieves an overall accuracy of 94%, demonstrating its reliability in classifying model outputs under steering. E. Steering Effects Discussion From the CRH perspective, commonly used linear criteria can be reinterpreted geometrically. Directional agreement effectively measures the consistency of the axis-aligned projection across tr...

    2025