Pith. sign in

REVIEW 2 major objections 1 minor 12 references

Code-switched inputs in multilingual models anchor to the grammar language, and steering hidden states toward the source anchor restores question-answering accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-26 18:11 UTC pith:E6CNPVYV

load-bearing objection Anchor Bias and CANVAS identify a grammar-frame pattern in CS representations and offer a workable steering fix, but the causal role of the geometric measure is not yet pinned down. the 2 major comments →

arxiv 2606.19668 v1 pith:E6CNPVYV submitted 2026-06-18 cs.CL

Code-Switching Reveals Language Anchoring in Multilingual LLMs

Jeonghyun Park , Seunghyun Yoon , Yonghyun Jun , Hwanhee Lee This is my paper
classification cs.CL
keywords code-switchingmultilingual LLMslanguage anchoringAnchor BiasCANVASquestion answeringinference-time interventiongrammar frame
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Multilingual models lose accuracy on sentences that mix languages compared with single-language inputs. The work treats grammar-forced code-switching as a controlled test to locate where mixed-language representations sit relative to each pure language. It finds a consistent pattern: when the grammar matches the source language the hidden states stay close to the source, while target grammar pulls them toward the target and produces larger drops in question-answering scores. The authors then build an inference-time method that pulls the hidden states back toward a source-derived anchor during the prefill stage. This intervention improves accuracy across several models and switching patterns without any retraining.

Core claim

Anchor Bias, a geometric distance that compares code-switched hidden states to their source-language and target-language counterparts, shows that source-framed code-switching remains source-anchored while target-framed code-switching shifts target-ward and correlates with greater QA degradation. CANVAS extracts a source-side canvas from the input and softly steers the hidden states of target-language tokens toward that anchor during prefill, recovering F1 scores across the tested models and conditions.

What carries the argument

Anchor Bias, the geometric measure of whether a code-switched hidden state lies closer to its source or target language counterpart, together with CANVAS, the inference-time method that steers hidden states toward an extracted source anchor.

Load-bearing premise

That the measured geometric distance between code-switched states and language anchors is the direct cause of the observed QA drop rather than a side effect, and that moving the states closer to the source anchor during prefill corrects the mismatch.

What would settle it

A controlled experiment in which CANVAS steering is applied to the same set of code-switched QA examples but produces no consistent rise in F1, or in which Anchor Bias values fail to rank-order the size of the degradation across models and grammar frames.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Source-framed code-switched inputs should retain higher QA accuracy than target-framed ones under the same model.
  • The grammar language of the switch, not the lexical content alone, determines the direction and strength of anchoring.
  • Soft steering of hidden states toward the source anchor during prefill should reduce degradation without changing model weights.
  • The same anchoring pattern and recovery should appear in other multilingual models beyond those tested.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Language identity may be represented as a separable geometric direction that can be adjusted independently of semantic content.
  • The same steering approach could be tested on mixed-language tasks other than QA, such as summarization or translation.
  • If anchoring proves causal, similar geometric interventions might address other forms of input mixing such as dialect shifts or register changes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The manuscript introduces Anchor Bias, a geometric measure quantifying whether code-switched (CS) hidden states in multilingual LLMs align closer to source or target language representations. It reports a consistent grammar-frame effect across MLLMs: source-framed CS remains source-anchored while target-framed CS shifts target-ward and exhibits larger QA degradation. Motivated by this pattern, the authors propose CANVAS, an inference-time intervention that extracts a source-side canvas and softly steers hidden states toward the source anchor during prefill, claiming consistent QA F1 recovery across models and CS conditions.

Significance. If the anchoring pattern is robust and CANVAS's gains are shown to arise specifically from correcting the measured geometric mismatch, the work supplies both a diagnostic for CS representational failures and a practical, inference-only mitigation. The cross-model consistency and use of grammar-forced CS as a controlled probe are positive features.

major comments (2)
  1. [Abstract] Abstract: the central inference that Anchor Bias captures the causal driver of QA degradation (rather than a correlated symptom of tokenization, attention, or lexical effects) is unsupported; the manuscript reports the correlation and CANVAS recovery but supplies no ablation that severs the geometric relationship while preserving other properties, nor a demonstration that steering magnitude predicts recovery beyond non-anchor baselines.
  2. [Abstract] Abstract: the claim that CANVAS 'consistently recovers QA F1 across MLLMs and CS conditions' by steering toward the source anchor requires evidence that the intervention outperforms generic steering or random shifts of comparable magnitude; without such controls the attribution to anchoring remains circular.
minor comments (1)
  1. The expansion of CANVAS is given but the precise extraction of the 'source-side canvas' and the steering formulation (e.g., interpolation coefficient, layer selection) are not summarized even at a high level, hindering immediate assessment of the method.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the constructive feedback on our manuscript. We address the two major comments regarding the causal interpretation of Anchor Bias and the controls for CANVAS. We will make revisions to clarify and strengthen these aspects.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central inference that Anchor Bias captures the causal driver of QA degradation (rather than a correlated symptom of tokenization, attention, or lexical effects) is unsupported; the manuscript reports the correlation and CANVAS recovery but supplies no ablation that severs the geometric relationship while preserving other properties, nor a demonstration that steering magnitude predicts recovery beyond non-anchor baselines.

    Authors: We agree that a direct causal demonstration via ablation would provide stronger evidence. The grammar-forced CS setting helps isolate the frame effect from lexical content, and the cross-model consistency of the anchoring pattern supports our interpretation. However, we lack an explicit ablation severing the geometry. In the revision, we will add a section discussing the inferential basis of our claims and include an analysis correlating steering magnitude with recovery gains. This addresses the comment partially through clarification and additional analysis. revision: partial

  2. Referee: [Abstract] Abstract: the claim that CANVAS 'consistently recovers QA F1 across MLLMs and CS conditions' by steering toward the source anchor requires evidence that the intervention outperforms generic steering or random shifts of comparable magnitude; without such controls the attribution to anchoring remains circular.

    Authors: The point is well-taken; without comparing to non-anchor steering methods, the specificity of the anchor target is not fully established. CANVAS extracts a source-side canvas from the actual input context, which is inherently tied to the anchoring measurement. To strengthen the attribution, we will incorporate baselines using random vector shifts and generic language direction steering of matched magnitude in the revised experiments. This will allow direct comparison of recovery effects. revision: yes

Circularity Check

0 steps flagged

No circularity: Anchor Bias and CANVAS are independently introduced without reduction to fitted inputs or self-citations

full rationale

The paper defines Anchor Bias as a new geometric measure of language anchoring in hidden states and observes its correlation with grammar-frame effects and QA degradation. CANVAS is then proposed as an inference-time steering method motivated by those observations. No equations, definitions, or claims reduce the reported effects or recovery to a parameter fitted from the same data, a self-citation chain, or an ansatz smuggled via prior work. The derivation remains self-contained against external benchmarks, with the measure and intervention having independent content.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 2 invented entities

Abstract provides no explicit free parameters, background axioms, or invented entities beyond the two new constructs introduced; full paper would be required to audit any hidden modeling choices.

invented entities (2)
  • Anchor Bias no independent evidence
    purpose: Geometric quantification of language anchoring in CS hidden states
    New measure defined in the paper to locate CS representations relative to source and target counterparts.
  • CANVAS no independent evidence
    purpose: Inference-time intervention that steers target-language hidden states toward a source anchor
    Proposed method motivated by the observed anchoring pattern.

pith-pipeline@v0.9.1-grok · 5728 in / 1223 out tokens · 31637 ms · 2026-06-26T18:11:09.464143+00:00 · methodology

0 comments
read the original abstract

Multilingual Large Language Models (MLLMs) are increasingly expected to handle Code-Switched (CS) inputs, yet mixing languages frequently degrades performance relative to source- or target-language monolingual counterparts. To understand this degradation, we use grammar-forced CS as a controlled diagnostic setting for locating CS representations relative to their source and target counterparts. We introduce Anchor Bias, a geometric measure that quantifies language anchoring, whether a CS hidden state aligns closer to its source or target language counterpart. Across diverse MLLMs, Anchor Bias reveals a consistent grammar-frame effect: source-framed CS stays source-anchored, whereas target-framed CS shifts target-ward and shows larger Question Answering (QA) degradation. Motivated by this representational pattern, we propose CANVAS (Contextual Anchor-based Neural Vector Alignment Steering), an inference-time intervention that extracts a source-side canvas from the input and softly steers target-language hidden states toward the source anchor during prefill. CANVAS consistently recovers QA F1 across MLLMs and CS conditions, showing that internal anchoring signals provide an actionable target for mitigating CS inference failures.

Figures

Figures reproduced from arXiv: 2606.19668 by Hwanhee Lee, Jeonghyun Park, Seunghyun Yoon, Yonghyun Jun.

Figure 1
Figure 1. Figure 1: Locating code-switched inputs in the mul￾tilingual representation space. Source- and target￾framed CS variants preserve different grammatical structures for the same information need, plotted along￾side their matched monolingual anchors. not random lexical noise but a structured lin￾guistic phenomenon governed by grammatical or￾ganization (Dogruöz et al. ˘ , 2021; Winata et al., 2023). Consequently, an MLL… view at source ↗
Figure 2
Figure 2. Figure 2: Per-layer anchor bias across model depth. Raw ABℓ on question-content tokens, averaged across examples, plotted against layer index for each model. Positive: source-anchored; negative: target-anchored. 4 Aligning Code-Switched Representations with CANVAS Motivated by our finding that target-ward anchor￾ing becomes increasingly target-anchored in the upper layers of target-framed code-switched in￾puts (Sect… view at source ↗
Figure 3
Figure 3. Figure 3: Latent displacement along the SRC–TGT axis. We project token states onto a 2D subspace where the horizontal axis captures the global SRC–TGT direction. Faded points indicate individual instances, large markers denote condition centroids, and arrows show the directional shift induced by CANVAS. correction mechanism. CANVAS further general￾izes to a Spanish source anchor, shows significant paired gains for m… view at source ↗
Figure 4
Figure 4. Figure 4: Adaptive intervention strength. CANVAS assigns larger interpolation strength α to more target￾heavy inputs, measured by lower source-token ratio ρ. resentational centroids for both GF-SRC and GF￾TGT undergo a substantial, rightward vector shift post-intervention, moving toward the clean source population space. This geometric relocation sup￾ports the interpretation that CANVAS shifts the latent processing … view at source ↗
Figure 5
Figure 5. Figure 5: Routing bias tracks representation anchor bias. Each point is an example-level CS input from an MoE model. The horizontal axis reports SRC-vs-TGT expert-routing bias, and the vertical axis reports rep￾resentation anchor bias. The positive association indi￾cates that expert allocation shifts in the same SRC/TGT direction as representation-space anchor bias. pairs: ES↔{Toba Batak, Bengali, French, Hindi, Kor… view at source ↗
Figure 6
Figure 6. Figure 6: Effective α distribution under the validation-selected CANVAS rule. We plot the area-normalized probability density of the effective interpolation strength used by CANVAS. We mark the validation-selected cen￾ter α0 = 0.45 and clip range [0.05, 0.75] with dashed and dotted vertical lines, respectively. We assign weaker updates to SRC-heavy GF-SRC examples and stronger updates to TGT-heavy GF-TGT examples, w… view at source ↗
Figure 7
Figure 7. Figure 7: Layer-wise CANVAS effect. We plot the magnitude of the source-ward shift across the relative position inside Lctrl. Darker cells indicate larger ∆ cosSRC or larger −∆ cosTGT. The effect is strongest near the later control layers, matching the layer-wise anchoring analysis. directly reflect the original Unicode surface form. We therefore decode each token ID back into its surface string before applying scri… view at source ↗
Figure 8
Figure 8. Figure 8: CANVAS success case: representation trajectory. Top: qualitative example showing the wrong base answer (2019) and the correct CANVAS answer (1660) for a Korean–English CS question. Bottom (a–c): layer￾by-layer hidden-state cosine similarity to the EN anchor (SRC) and KO anchor (TGT) for the CS prompt, and the resulting anchor bias AB= cos(hEN, hCS) − cos(hKO, hCS). In the base run, AB stays negative throug… view at source ↗
Figure 9
Figure 9. Figure 9: CANVAS failure case: representation trajectory. Top: qualitative example where direct CS answering returns the gold year (2018), while CANVAS outputs a wrong year (2017). Bottom (a–c): layer-by-layer cosine similarity to the EN anchor (SRC), similarity to the Spanish anchor (TGT), and anchor bias AB= cos(hEN, hCS)− cos(hES, hCS). CANVAS moves the CS state toward the SRC side but does not produce a stable f… view at source ↗
Figure 10
Figure 10. Figure 10: Source-ward representation movement. Signed projection ratio η of the CANVAS displace￾ment onto the SRC–TGT axis (η > 0: source-ward; η < 0: target-ward). H.12 Cumulative Recovery Trajectory Setup. The per-layer projection ηℓ measures the fraction of the SRC–TGT representational gap that CANVAS closes at control layer ℓ. Reading these layer-wise contributions cumulatively answers a 28 [PITH_FULL_IMAGE:fi… view at source ↗
Figure 11
Figure 11. Figure 11: Model-wise CANVAS representation movement. We visualize the same representation analysis for four representative models. Each row is a model and each column is a grammar-forced CS condition. Triangles mark SRC/TGT anchors, gray points mark base CS states, teal points mark CANVAS states, and arrows connect the base and CANVAS centroids. Across models, CANVAS shifts the CS centroid toward the source-side re… view at source ↗
Figure 12
Figure 12. Figure 12: Model-wise source-ward representation movement. We plot the signed projection ratio η of the CAN￾VAS displacement onto the SRC–TGT anchor axis for each analyzed model. Positive values indicate movement toward the source anchor. Each panel reports the percentage of examples with η > 0. 35 [PITH_FULL_IMAGE:figures/full_fig_p035_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Model-wise adaptive intervention strength. We compare the source-token ratio ρ with the effective interpolation strength α selected by CANVAS for each analyzed model. Lower ρ corresponds to more target-heavy inputs, and the binned trend lines show how the adaptive rule changes intervention strength within each model. 36 [PITH_FULL_IMAGE:figures/full_fig_p036_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

12 extracted references · 6 canonical work pages · 1 internal anchor

  1. [1]

    Yuxin Chen, Zhengzhou Cai, Xiangtian Ji, Weixiang Zhao, An Zhang, Xiang Wang, and Tat-Seng Chua

    Multilingual hallucination gaps in large lan- guage models.arXiv preprint arXiv:2410.18270. Yuxin Chen, Zhengzhou Cai, Xiangtian Ji, Weixiang Zhao, An Zhang, Xiang Wang, and Tat-Seng Chua

    work page arXiv

  2. [2]

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others

    Understanding multilingualism in mixture- of-experts llms: Routing mechanism, expert spe- cialization, and layerwise steering.arXiv preprint arXiv:2601.14050. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the fronti...

    work page arXiv 2025

  3. [3]

    Muhammad Huzaifah, Weihua Zheng, Nattapol Chan- paisit, and Kui Wu

    Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generaliza- tion.Preprint, arXiv:2003.11080. Muhammad Huzaifah, Weihua Zheng, Nattapol Chan- paisit, and Kui Wu. 2024. Evaluating code- switching translation with large language models. In Proceedings of LREC-COLING, pages 6381–6394. Albert Q. Jiang, Alexandre Sablayrolles,...

    work page arXiv 2003

  4. [4]

    Shaoyang Xu, Junzhuo Li, and Deyi Xiong

    Not all languages are equal: Insights into multilingual retrieval-augmented generation.arXiv preprint arXiv:2410.21970. Shaoyang Xu, Junzhuo Li, and Deyi Xiong. 2023. Lan- guage representation projection: Can we transfer factual knowledge across languages in multilingual language models? InProceedings of the 2023 Con- ference on Empirical Methods in Natur...

    work page arXiv 2023

  5. [5]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388. Zheng-Xin Yong, Ruochen Zhang, Jessica Zosa Forde, Skyler Wang, Arjun Subramonian, Holy Lovenia, Samuel Cahyawijaya, Genta Indra Winata, Lintang Sutawika, Jan Christian Blaise Cruz, Yin Lin Tan, Long Phan, Rowena Garcia, Thamar Solorio, and Alham Fikri Aji. 2023. Prompting multilingual large language...

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  6. [6]

    Section B describes the use of LLMs during manuscript preparation

  7. [7]

    Section C providesimplementation details, including the OpenRouter pipeline, local de- coding setup, hidden-state extraction, quantiza- tion fallback, and hardware resources

  8. [8]

    Section D providescomplementary analyses for MoE models, including MoE routing ex- traction, routing-profile similarity, routing con- figuration, Cohen’sdfor routing separation, and the relationship between expert routing and representation-level anchor bias

  9. [9]

    Section E testssource-language generaliza- tion with Spanish as the source anchor, cov- ering both the Spanish-pivot anchor-bias repli- cation and Spanish-anchor CANVAS results

  10. [10]

    Section F describes thedataset construction and matched comparison statistics, caveats on translated target-language questions, addi- tional code-switching conditions, and prompt templates

  11. [11]

    Section G provides additionalanchor-bias analyses, including anisotropy diagnostics, language-wise breakdowns, metric variants, frontier-model scaling validation, and same- lexicon order counterfactuals that disentangle lexical language from word order

  12. [12]

    Who wrote Pride and Prejudice?

    Section H provides detailed analyses ofCAN- V AS, including the adaptiveαschedule, layer-wise effects, source-heavy checks, token- level language tagging, hidden-state extraction, layer-span ablations, language-wise mitigation, case studies, model-wise movement diagnos- tics, cost analysis, additional movement anal- yses, cumulative recovery trajectories,...

    work page arXiv 2025