X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model
Pith reviewed 2026-05-12 14:51 UTC · model grok-4.3
The pith
Separate learnable embeddings for each data source let a standard transformer handle heterogeneous robot data as a generalist vision-language-action model.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
A soft-prompted transformer uses separate sets of learnable embeddings for each distinct data source to serve as embodiment-specific prompts; these prompts, together with the shared transformer, enable effective exploitation of cross-embodiment features in large heterogeneous datasets, allowing a 0.9B model to reach state-of-the-art performance across simulation and real-world benchmarks for dexterity and adaptation.
What carries the argument
The soft-prompt mechanism of separate learnable embedding sets infused into the transformer for each data source, which conditions the model on embodiment while keeping the main network shared and scalable.
Load-bearing premise
Separate learnable embeddings per data source can capture and exploit cross-embodiment differences while the shared transformer learns general features without interference or the need for more complex conditioning.
What would settle it
An ablation that removes the per-data-source embedding sets and measures whether performance on cross-embodiment adaptation and dexterity tasks drops to match or fall below non-prompted baselines.
read the original abstract
Successful generalist Vision-Language-Action (VLA) models rely on effective training across diverse robotic platforms with large-scale, cross-embodiment, heterogeneous datasets. To facilitate and leverage the heterogeneity in rich, diverse robotic data sources, we propose a novel Soft Prompt approach with minimally added parameters, by infusing prompt learning concepts into cross-embodiment robot learning and introducing separate sets of learnable embeddings for each distinct data source. These embeddings serve as embodiment-specific prompts, which in unity empower VLA models with effective exploitation of varying cross-embodiment features. Our new X-VLA, a neat flow-matching-based VLA architecture, relies exclusively on soft-prompted standard Transformer encoders, enjoying both scalability and simplicity. Evaluated across 6 simulations as well as 3 real-world robots, our 0.9B instantiation-X-VLA-0.9B simultaneously achieves SOTA performance over a sweep of benchmarks, demonstrating superior results on a wide axes of capabilities, from flexible dexterity to quick adaptation across embodiments, environments, and tasks. Website: https://thu-air-dream.github.io/X-VLA/
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces X-VLA, a flow-matching-based Vision-Language-Action architecture that augments standard Transformer encoders with soft prompts consisting of separate learnable embeddings per data source. These embodiment-specific prompts are intended to allow the shared transformer to learn general cross-embodiment features while handling dataset heterogeneity with minimal added parameters. The 0.9B model is reported to achieve simultaneous SOTA results across 6 simulation and 3 real-world benchmarks, with claimed strengths in dexterity and rapid adaptation.
Significance. If the performance claims and the causal role of the soft-prompt mechanism are substantiated, the work would provide a lightweight, scalable route to generalist VLAs that avoids bespoke conditioning modules. The reliance on unmodified Transformer encoders and the broad benchmark sweep are positive features that could influence subsequent cross-embodiment policy research.
major comments (2)
- [Abstract, §3, §4] The central SOTA claim in the abstract and §1 rests on the untested assumption that per-source soft prompts isolate embodiment features without interference in the shared transformer. No controlled ablation (e.g., unified embedding baseline versus per-source prompts) appears in §4 or §5, leaving the performance gains potentially attributable to data scale, flow-matching, or other unisolated factors rather than the proposed design.
- [§4.2, Table 2] §4.2 and Table 2: quantitative results are presented without error bars, statistical significance tests, or per-task breakdowns that would allow assessment of whether the reported superiority holds uniformly across the 9 benchmarks or is driven by a subset of tasks.
minor comments (2)
- [§3.1] The integration of soft prompts into the transformer (prepending, additive bias, or attention masking) is described at a high level in §3.1; a precise equation or pseudocode would improve reproducibility.
- [Figure 2] Figure 2 caption and axis labels could be expanded to clarify which curves correspond to which embodiment prompts.
Simulated Author's Rebuttal
We thank the referee for their constructive and insightful comments, which have helped us identify areas to strengthen the manuscript. We respond to each major comment below and commit to revisions that address the concerns raised.
read point-by-point responses
-
Referee: [Abstract, §3, §4] The central SOTA claim in the abstract and §1 rests on the untested assumption that per-source soft prompts isolate embodiment features without interference in the shared transformer. No controlled ablation (e.g., unified embedding baseline versus per-source prompts) appears in §4 or §5, leaving the performance gains potentially attributable to data scale, flow-matching, or other unisolated factors rather than the proposed design.
Authors: We agree that a direct controlled ablation isolating the contribution of per-source soft prompts versus a unified embedding baseline would provide stronger causal evidence for the design choice. While the manuscript shows that X-VLA outperforms prior cross-embodiment methods and that the soft-prompt approach adds minimal parameters, we acknowledge that the current experiments do not fully rule out contributions from data scale or the flow-matching objective. In the revised manuscript, we will add a new ablation study that trains a unified-embedding variant under identical conditions (same data, architecture, and training procedure) and directly compares it to the per-source prompt version. This will clarify the role of embodiment-specific prompts in managing dataset heterogeneity. revision: yes
-
Referee: [§4.2, Table 2] §4.2 and Table 2: quantitative results are presented without error bars, statistical significance tests, or per-task breakdowns that would allow assessment of whether the reported superiority holds uniformly across the 9 benchmarks or is driven by a subset of tasks.
Authors: We appreciate this feedback on result presentation. In the revised manuscript, we will update Table 2 to include error bars (standard deviation across multiple random seeds for key experiments). We will also expand the results section with per-task performance breakdowns, either as an additional table or in the appendix, to demonstrate consistency across the 9 benchmarks. Where computationally feasible, we will report statistical significance tests (e.g., paired t-tests against baselines) to support the SOTA claims. These changes will make the quantitative evidence more robust and transparent. revision: yes
Circularity Check
No circularity in architectural proposal or performance claims
full rationale
The paper introduces an architectural modification—separate learnable soft-prompt embeddings per data source within a standard flow-matching transformer encoder—presented as an empirical design choice to handle cross-embodiment heterogeneity. No equations, derivations, or first-principles results are claimed that reduce by construction to fitted parameters, self-definitions, or prior self-citations. SOTA results are reported from direct benchmark evaluations across simulation and real robots rather than from any closed-loop prediction or uniqueness theorem. The central mechanism (prompts isolating source-specific features while the shared transformer learns general ones) is an ansatz justified by design intuition and results, not by any load-bearing self-referential step or imported theorem. This is a standard non-circular model proposal.
Axiom & Free-Parameter Ledger
Forward citations
Cited by 60 Pith papers
-
LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Models
LIBERO-Safety supplies a scalable benchmark, data-generation pipeline, and 19,664-demonstration dataset that exposes a generalization-safety tension in current VLA models where diverse training improves collision avoi...
-
Cloak: Zero-Shot Cross-Embodiment Manipulation by Masking the End-Effector from the VLA
Masking the end-effector from wrist views during training lets a single-gripper VLA transfer zero-shot to other grippers, arms, and five-fingered hands while keeping original performance.
-
Frequency-Aware Flow Matching for Continuous and Consistent Robotic Action Generation
FAFM performs flow matching in the frequency domain using DCT on action sequences to produce continuous temporally consistent robotic actions with a Sobolev-style smoothness regularizer.
-
EBench: Elemental Diagnosis of Generalist Mobile Manipulation Policies
EBench is a benchmark that evaluates generalist mobile manipulation policies on 26 tasks across 5 capability and 4 generalization dimensions, revealing distinct capability profiles among models with similar success rates.
-
ThinkingVLA: Interleaved Vision and Language Reasoning for Robotic Manipulation
ThinkingVLA is a Mixture-of-Transformers VLA model that performs interleaved forward CoT for subgoal and image prediction followed by inverse CoT grounded on the predicted image to generate actions.
-
DuoBench: A Reproducible Benchmark for Bimanual Manipulation in Simulation and the Real World
DuoBench introduces eleven bimanual manipulation tasks with stage-based evaluation and human datasets to benchmark imitation-learning and vision-language-action policies on dual-arm robots in sim and real settings.
-
X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretraining
X-Tokenizer creates semantic action tokens via asymmetric residual quantization and contrastive pretraining on large trajectory data, outperforming prior methods like FAST on robotic tasks.
-
ActionMap: Robot Policy Learning via Voxel Action Heatmap
ActionMap introduces a voxel heatmap action head for VLA models that improves policy learning by exploiting geometric structure in the action space.
-
Can VLA Models Learn from Real-World Data Continually without Forgetting?
VLA models exhibit catastrophic forgetting on a new real-world dataset of four sequential manipulation tasks, with experience replay implementation factors evaluated for mitigation.
-
RotVLA: Rotational Latent Action for Vision-Language-Action Model
RotVLA models latent actions as continuous SO(n) rotations with triplet-frame supervision and flow-matching to reach 98.2% success on LIBERO and 89.6%/88.5% on RoboTwin2.0 using a 1.7B-parameter model.
-
See What Matters: Differentiable Grid Sample Pruning for Generalizable Vision-Language-Action Model
GridS is a plug-and-play differentiable module for geometry-aware visual token resampling in VLA models that achieves under 10% token retention and 76% FLOPs reduction with no success-rate loss.
-
RIO: Flexible Real-Time Robot I/O for Cross-Embodiment Robot Learning
RIO introduces a lightweight open-source framework that abstracts real-time robot I/O to support easy switching between embodiments and platforms for collecting data and deploying VLAs.
-
ALAM: Algebraically Consistent Latent Action Model for Vision-Language-Action Models
ALAM creates algebraically consistent latent action transitions from videos to act as auxiliary generative targets, raising robot policy success rates from 47.9% to 85.0% on MetaWorld MT50 and 94.1% to 98.1% on LIBERO.
-
One Token Per Frame: Reconsidering Visual Bandwidth in World Models for VLA Policy
Reducing visual input to one token per frame in VLA world models maintains or improves long-horizon performance on MetaWorld, LIBERO, and real-robot tasks.
-
OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation
OA-WAM uses persistent address vectors and dynamic content vectors in object slots to enable addressable world-action prediction, improving robustness on manipulation benchmarks under scene changes.
-
VLA-GSE: Boosting Parameter-Efficient Fine-Tuning in VLA with Generalized and Specialized Experts
VLA-GSE improves VLA adaptation by initializing generalized shared experts and specialized routed experts via spectral decomposition of the backbone, outperforming full fine-tuning and other PEFT methods on robotic be...
-
MolmoAct2: Action Reasoning Models for Real-world Deployment
MolmoAct2 delivers an open VLA model with new specialized components, datasets, and techniques that outperforms baselines on benchmarks while releasing all weights, code, and data for real-world robot use.
-
Being-H0.7: A Latent World-Action Model from Egocentric Videos
Being-H0.7 adds future-aware latent reasoning to direct VLA policies via dual-branch alignment on latent queries, matching world-model benefits at VLA efficiency.
-
CF-VLA: Efficient Coarse-to-Fine Action Generation for Vision-Language-Action Policies
CF-VLA uses a coarse initialization over endpoint velocity followed by single-step refinement to achieve strong performance with low inference steps on CALVIN, LIBERO, and real-robot tasks.
-
Modular Sensory Stream for Integrating Physical Feedback in Vision-Language-Action Models
MoSS augments VLAs with decoupled modality streams for multiple physical signals, achieving synergistic gains in real-world robot tasks via joint attention and auxiliary future-signal prediction.
-
${\pi}_{0.7}$: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities
π₀.₇ is a steerable generalist robotic model that uses rich multimodal prompts including language, subgoal images, and performance metadata to achieve out-of-the-box generalization across tasks and robot bodies.
-
QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Models
QuantVLA is the first post-training quantization framework for VLA models that quantizes the diffusion transformer action head and reports higher task success rates than full-precision baselines with roughly 70% memor...
-
VLA-Corrector: Lightweight Detect-and-Correct Inference for Adaptive Action Horizon
VLA-Corrector adds a detect-and-correct inference layer using a latent vision monitor and online gradient guidance to enable adaptive action horizons in chunked VLA policies.
-
UniTacVLA: Unified Tactile Understanding and Prediction in Vision Language Action Models
UniTacVLA builds a state-aware and dynamics-aware tactile prior via unified latent space, tactile chain-of-thought, and mixed real/predicted feedback controller to boost dexterous manipulation performance.
-
Training Vision-Language-Action Models with Dense Embodied Chain-of-Thought Supervision
ZR-0 trains a dual-stream VLA model on 60 million frames with dense ECoT annotations so that high-level reasoning transfers across single-arm, bimanual, and humanoid embodiments while skipping reasoning at inference.
-
Training Vision-Language-Action Models with Dense Embodied Chain-of-Thought Supervision
ZR-0 is a dual-stream VLA model trained with dense ECoT supervision on 60M frames from 400K trajectories to enable cross-embodiment transfer in simulation and real-world settings.
-
Critical Interval MSE: Toward Reliable Offline Validation for Robot Manipulation Policies
CI-MSE improves Spearman's rank correlation between offline validation error and real rollout performance from -0.61 (raw MSE) to -0.87 across policy checkpoints in simulation and real-world robot manipulation experiments.
-
DIM-WAM: World-Action Modeling with Diverse Historical Event Memory
DiM-WAM is a memory-augmented world-action model that integrates multi-scale historical events and global task progress to improve long-horizon robot manipulation performance.
-
SPACE: Enabling Learning from Cross-Robot Data Toward Generalist Policies
SPACE trains a Cartesian state-delta policy plus an Action Adapter to learn generalist robot policies from cross-embodiment and cross-hardware data while remaining robust to dynamics changes.
-
LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Models
Introduces LIBERO-Safety benchmark with parametric scenario generation and 19,664 collision-free demonstrations, then evaluates VLA models to reveal a generalization-safety tension.
-
UniFS: Unified Fast-to-Slow Hierarchical Architecture for Vision-Language-Action Models
UniFS achieves 98.3% success on LIBERO with 2.1x lower latency than prior fast-slow VLA models by stratifying VLM layer update frequencies, inverting latent interactions, and applying multi-level supervision.
-
EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies
EventVLA introduces foundational visual anchors and a Keyframe Evidence Memory module that predicts future keyframe probabilities from VLA embeddings to improve long-horizon task success by an average of 40% on 17 sim...
-
Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models
Qwen-RobotManip applies unified alignment across representation, motion, and behavior to enable large-scale training on heterogeneous manipulation data, yielding emergent generalization on out-of-distribution robotic ...
-
ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining
ACE-Ego-0 is a VLA pretraining framework that turns egocentric human videos into robot-format pseudo-actions via a video-to-action pipeline and trains jointly with robot data under a reliability-aware objective.
-
EmbodiSteer: Steering Embodiment-Agnostic Visuomotor Policies with Joint-Space Guidance for Zero-Shot Cross-Embodiment Deployment
EmbodiSteer steers embodiment-agnostic Cartesian diffusion policies into joint space with Jacobian-based collision guidance after each denoising step for zero-shot cross-embodiment deployment.
-
APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
APT pretrains the action expert as a vision-action prior on frozen VLM features then adds language through gated fusion to improve OOD instruction generalization in continuous-action VLA policies.
-
DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action model
DAM-VLA decouples per-modality temporal processing in vision-language-action models via latent buffers refreshed at sensor rates, achieving 95.2% average success versus 40.95% for synchronous baselines on seven real-w...
-
$\mu$VLA: On Recurrent Memory for Partially Observable Manipulation in VLA Models
Adding recurrent memory tokens to VLA models raises success rates on partially observable manipulation tasks from 0.42 to 0.84 on training and 0.07 to 0.23 on held-out tasks while preserving performance under full obs...
-
Next Forcing: Causal World Modeling with Multi-Chunk Prediction
Next Forcing augments video generation models with auxiliary multi-chunk prediction modules to achieve faster training convergence, higher accuracy at high frame rates, and 2x faster inference on world modeling benchmarks.
-
SARM2: Multi-Task Stage Aware Reward Modeling for Self Improving Robotic Manipulation
SARM2 presents RM, a multi-task stage-aware reward model achieving 80% lower value-estimation MSE, which when used in SPIRAL boosts manipulation task success from ~50% to near-perfect on several benchmarks.
-
GEAR-VLA: Learning Geometry-Aware Action Representations for Generalizable Robotic Manipulation
GEAR-VLA learns geometry-aware action representations via coarse-to-fine pretraining, gradient-decoupled DiT action expert, semantic-aligned 3D integration, and embodiment canonicalization, reporting SOTA results on L...
-
AxisGuide: Grounding Robot Action Coordinate System in RGB Observations for Robust Visuomotor Manipulation
AxisGuide augments RGB images with rendered robot base-frame axis cues to improve generalization of visuomotor manipulation policies under distribution shifts.
-
TempoVLA: Learning Speed-Controllable Vision-Language-Action Policies
TempoVLA learns a single VLA policy with controllable execution speed via variable-speed trajectory augmentation and explicit speed conditioning.
-
AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding
AffordanceVLA proposes a VLA model with affordance-aware modules (Which2Act, Where2Act, How2Act) in a Mixture-of-Transformer trained in three stages to improve robotic manipulation.
-
Flash-WAM: Modality-Aware Distillation for World Action Models
Flash-WAM introduces modality-specific consistency parametrizations to distill joint video-action diffusion models to single-step inference, delivering 23x speedup with preserved benchmark performance.
-
Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation
ERVLA trains on a 978k-trajectory embodied CoT corpus using reasoning as supervision with dropout, then predicts actions without CoT at test time, reaching 86.9% on LIBERO-Plus and 53.2% on VLABench.
-
See Less, Specify More: Visual Evidence Budgets for Generalizable VLAs
S2 improves generalization in vision-language-action models by using goal-preserving refined language guidance and explicit visual evidence budgets, raising mean subtask success from 54.2% to 79.0% on eight real-robot...
-
Closed-Loop Neural Activation Control in Vision-Language-Action Models
CTRL-STEER applies PID or RL-based feedback control to adaptively steer motion-aligned residual directions in VLA models, yielding more stable regulation and better task success on LIBERO benchmarks than fixed steering.
-
Continuous Reasoning for Vision-Language-Action
Continuous Reasoning for VLA introduces a shared Gaussian latent for continuous thoughts, trained with self-verification to improve action prediction on LIBERO-PRO and real robots.
-
3DVLA: Enhancing Vision-Language-Action Models via 3D Spatial and Instance Understanding
3DVLA is a plug-and-play framework that enhances pretrained VLAs with pervasive 3D feature encoding using multi-view consistency and Spatially-Conditioned Geometry Aggregation, an instance estimation module, and a mas...
-
X-DiffVLA: X-Embodied Diffusion Action Heads for Vision-Language-Action Models
X-DiffVLA proposes a diffusion VLA model using Embodiment Forcing and Morphological Tree Diffusion to achieve SOTA cross-embodied performance on simulation benchmarks with 15.3% and 12.5% gains.
-
DEFLECT: Delay-Robust Execution via Flow-matching Likelihood-Estimated Counterfactual Tuning for VLA Policies
DEFLECT is an offline post-training method that improves async VLA policy success rates under high inference delays by using flow-matching likelihood ratios on counterfactual fresh/stale action pairs from a frozen ref...
-
Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models
GTA-VLA conditions VLA models on user spatial priors to produce a unified spatial-visual chain-of-thought, reaching 81.2% success on SimplerEnv WidowX and improving performance under out-of-distribution shifts.
-
See What Matters: Differentiable Grid Sample Pruning for Generalizable Vision-Language-Action Model
GridS reduces visual tokens in VLA models to under 10% of the original count via task-aware differentiable resampling, delivering 76% lower FLOPs with no drop in task success rate on benchmarks and real robots.
-
RoboMemArena: A Comprehensive and Challenging Robotic Memory Benchmark
RoboMemArena is a new large-scale robotic memory benchmark with real-world tasks, and PrediMem is a dual VLA system that outperforms baselines by managing memory buffers with predictive coding.
-
Unified Noise Steering for Efficient Human-Guided VLA Adaptation
UniSteer unifies human corrective actions and noise-space RL for VLA adaptation by inverting actions to noise targets, raising success rates from 20% to 90% in 66 minutes across four real-world manipulation tasks.
-
ALAM: Algebraically Consistent Latent Action Model for Vision-Language-Action Models
ALAM introduces algebraic consistency regularization on latent action transitions from videos, raising VLA success rates from 47.9% to 85.0% on MetaWorld MT50 and 94.1% to 98.1% on LIBERO.
-
One Token Per Frame: Reconsidering Visual Bandwidth in World Models for VLA Policy
Reducing visual input to one token per frame in world models for vision-language-action policies maintains long-horizon performance while improving success rates on MetaWorld, LIBERO, and real-robot tasks.
-
One Token Per Frame: Reconsidering Visual Bandwidth in World Models for VLA Policy
Reducing visual input to one token per frame via adaptive attention pooling and a unified flow-matching objective improves long-horizon performance in VLA policies on MetaWorld, LIBERO, and real-robot tasks.
-
BioProVLA-Agent: An Affordable, Protocol-Driven, Vision-Enhanced VLA-Enabled Embodied Multi-Agent System with Closed-Loop-Capable Reasoning for Biological Laboratory Manipulation
Presents BioProVLA-Agent, a protocol-driven VLA-enabled multi-agent system for embodied biological manipulation with visual state verification and AugSmolVLA augmentation for robustness in wet-lab conditions.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.