ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving

Jingru Luo; Liu Ren; Sikai Chen; Xin Ye; Zihao Sheng

REVIEW 2 major objections 1 minor 2 cited by

Augmenting VLA driving models with future image prediction supplies both dense supervision and an uncertainty signal for safe policy exploration.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-05-13 20:49 UTC pith:QGOCE4Q3

load-bearing objection ExploreVLA folds dense future RGB and depth prediction into a VLA driving model for richer supervision and uncertainty-driven exploration with a safety gate, but the evidence tying uncertainty to safe novelty is thin. the 2 major comments →

arxiv 2604.02714 v2 pith:QGOCE4Q3 submitted 2026-04-03 cs.CV

ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving

Zihao Sheng , Xin Ye , Jingru Luo , Sikai Chen , Liu Ren This is my paper

classification cs.CV

keywords autonomous drivingvision-language-actionworld modelingreinforcement learningexplorationimage predictionpolicy optimizationend-to-end driving

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

End-to-end autonomous driving models based on vision-language-action architectures are limited by imitation learning to replicating expert behaviors and fail in novel situations. The paper establishes that adding a world model to predict future RGB and depth images provides dense supervision that enriches the planning backbone with fine-grained visual and geometric features. This same world model supplies an intrinsic reward based on image prediction uncertainty, which flags out-of-distribution trajectories as learning opportunities when gated by safety. The policy is then optimized via Group Relative Policy Optimization. A sympathetic reader would care because the method offers a concrete way to introduce reinforcement learning exploration into offline-trained driving systems without requiring direct state transitions.

Core claim

The central claim is that a unified understanding-and-generation framework, where the VLA model augments trajectory prediction with future RGB and depth image generation, creates dense world modeling objectives that both enrich representations for planning and generate an uncertainty-based intrinsic reward for exploration, which when safety-gated enables Group Relative Policy Optimization to produce more robust driving policies.

What carries the argument

The dense world model for generating future RGB and depth images, whose prediction uncertainty measures a trajectory's novelty relative to the training distribution to supply the intrinsic reward.

Load-bearing premise

The world model's image prediction uncertainty reliably indicates both novelty and safety, allowing the safety-gated reward to produce valuable exploration without unsafe behaviors or training instability.

What would settle it

If removing the uncertainty-based exploration reward causes no drop in performance on out-of-distribution test scenarios compared to pure imitation learning, the claim that uncertainty supplies useful exploration signals would be falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

ExploreVLA folds dense future RGB and depth prediction into a VLA driving model for richer supervision and uncertainty-driven exploration with a safety gate, but the evidence tying uncertainty to safe novelty is thin.

read the letter

The main thing here is that the authors add future image and depth generation as an auxiliary task inside the VLA backbone. This gives denser supervision that should improve geometric and visual features for planning, and they then use the same model's prediction uncertainty as an intrinsic reward for Group Relative Policy Optimization, filtered by a safety gate. They report 93.7 PDMS and 88.8 EPDMS on NAVSIM, which is competitive on those benchmarks. The unification is the clean part: one set of predictions serves both supervision and the exploration signal without extra networks. That is a reasonable way to move past pure imitation learning while staying inside the VLA setup. The paper builds sensibly on existing world-model and RL ideas for driving and keeps the architecture straightforward. The soft spots are around the central assumption. The claim that uncertainty reliably flags useful but safe out-of-distribution states rests on the world model being accurate enough to separate drivable novelty from dangerous cases, yet there is no reported correlation analysis, no ablation that turns the exploration reward on and off, and no inspection of failure trajectories. Without those, it is hard to know whether the benchmark gains come from the added supervision alone or from the RL component, and whether the safety gate actually prevents bad exploration. The circularity risk is modest because the reward uses held-out trajectories, but it still depends on the same learned model. This is for people already working on VLA or world-model approaches to end-to-end driving who want a concrete example of adding exploration inside one network. A reader could pull the dense prediction trick and test it themselves. It deserves peer review because the integration is new enough and the benchmarks matter, even though the paper would need stronger checks on the uncertainty signal before the claims are fully convincing. I would send it to referees rather than desk reject.

Referee Report

2 major / 1 minor

Summary. The manuscript introduces ExploreVLA, a unified VLA framework for end-to-end autonomous driving that augments trajectory prediction with future RGB and depth image generation as dense world-modeling objectives. These objectives supply both enriched supervision for the planning backbone and an intrinsic reward signal based on the model's own image-prediction uncertainty, which is combined with a safety gate and optimized via Group Relative Policy Optimization (GRPO). Experiments are reported on NAVSIM and nuScenes, with state-of-the-art PDMS of 93.7 and EPDMS of 88.8 claimed on NAVSIM.

Significance. If the uncertainty-based exploration mechanism can be shown to identify safe novelty without reinforcing unsafe behaviors, the work would meaningfully advance RL-augmented VLA driving by addressing the distribution-shift brittleness of pure imitation learning. The dual use of dense prediction for both representation learning and intrinsic reward is a conceptually clean contribution.

major comments (2)

[Abstract] Abstract: The central claim that image-prediction uncertainty serves as a reliable, safety-gated intrinsic reward for valuable exploration is load-bearing yet unsupported; no correlation analysis between uncertainty and out-of-distribution but drivable states, no ablation of the safety gate, and no failure-case inspection are provided, leaving open the possibility that high uncertainty simply flags imminent collisions or rule violations.
[Abstract] Abstract / Experiments: The reported SOTA PDMS 93.7 and EPDMS 88.8 scores are presented without ablations, baseline comparisons, error bars, or details on how the safety gate was implemented, making it impossible to determine whether the gains derive from the proposed exploration signal or from post-hoc tuning.

minor comments (1)

[Abstract] The manuscript states that code and a demo will be released but supplies insufficient methodological detail (e.g., exact form of the uncertainty reward, GRPO hyperparameters, or safety-gate threshold) for independent verification in the current version.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We agree that the central claims require stronger empirical support and have revised the manuscript to include the requested analyses, ablations, and implementation details.

read point-by-point responses

Referee: [Abstract] Abstract: The central claim that image-prediction uncertainty serves as a reliable, safety-gated intrinsic reward for valuable exploration is load-bearing yet unsupported; no correlation analysis between uncertainty and out-of-distribution but drivable states, no ablation of the safety gate, and no failure-case inspection are provided, leaving open the possibility that high uncertainty simply flags imminent collisions or rule violations.

Authors: We acknowledge the need for direct validation of the uncertainty signal. In the revised version we add a correlation analysis between image-prediction uncertainty and out-of-distribution yet drivable states, an ablation that removes the safety gate, and qualitative inspection of failure cases demonstrating that high-uncertainty trajectories correspond to safe novel scenarios rather than imminent collisions. revision: yes
Referee: [Abstract] Abstract / Experiments: The reported SOTA PDMS 93.7 and EPDMS 88.8 scores are presented without ablations, baseline comparisons, error bars, or details on how the safety gate was implemented, making it impossible to determine whether the gains derive from the proposed exploration signal or from post-hoc tuning.

Authors: We agree that the experimental presentation must be strengthened. The revised manuscript includes component-wise ablations (world-modeling loss, uncertainty reward, safety gate), additional baseline comparisons, error bars from multiple random seeds, and a precise description of the safety-gate logic and thresholds to isolate the contribution of the exploration signal. revision: yes

Circularity Check

0 steps flagged

Derivation chain is self-contained with no circular reductions

full rationale

The paper trains a unified VLA world model on dense RGB/depth prediction objectives to enrich the planning backbone, then deploys the same model's image-prediction uncertainty as an intrinsic reward inside a safety-gated GRPO loop. This construction follows the standard curiosity-driven exploration pattern and does not equate the final PDMS/EPDMS benchmark scores to the training reward by definition; the benchmarks are computed on independent held-out trajectories after policy optimization. No load-bearing self-citation, uniqueness theorem, or fitted-input-renamed-as-prediction step appears in the derivation. The central empirical claims therefore rest on external benchmark evaluation rather than internal definitional closure.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no explicit free parameters, axioms, or invented entities; the approach implicitly relies on standard assumptions from world models and RL that uncertainty correlates with novelty and that safety gating prevents harm.

pith-pipeline@v0.9.0 · 5593 in / 1119 out tokens · 48683 ms · 2026-05-13T20:49:17.071003+00:00 · methodology

0 comments

read the original abstract

End-to-end autonomous driving models based on Vision-Language-Action (VLA) architectures have shown promising results by learning driving policies through behavior cloning on expert demonstrations. However, imitation learning inherently limits the model to replicating observed behaviors without exploring diverse driving strategies, leaving it brittle in novel or out-of-distribution scenarios. Reinforcement learning (RL) offers a natural remedy by enabling policy exploration beyond the expert distribution. Yet VLA models, typically trained on offline datasets, lack directly observable state transitions, necessitating a learned world model to anticipate action consequences. In this work, we propose a unified understanding-and-generation framework that leverages world modeling to simultaneously enable meaningful exploration and provide dense supervision. Specifically, we augment trajectory prediction with future RGB and depth image generation as dense world modeling objectives, requiring the model to learn fine-grained visual and geometric representations that substantially enrich the planning backbone. Beyond serving as a supervisory signal, the world model further acts as a source of intrinsic reward for policy exploration: its image prediction uncertainty naturally measures a trajectory's novelty relative to the training distribution, where high uncertainty indicates out-of-distribution scenarios that, if safe, represent valuable learning opportunities. We incorporate this exploration signal into a safety-gated reward and optimize the policy via Group Relative Policy Optimization (GRPO). Experiments on the NAVSIM and nuScenes benchmarks demonstrate the effectiveness of our approach, achieving a state-of-the-art PDMS score of 93.7 and an EPDMS of 88.8 on NAVSIM. The code is available at https://zihaosheng.github.io/ExploreVLA/.

Figures

Figures reproduced from arXiv: 2604.02714 by Jingru Luo, Liu Ren, Sikai Chen, Xin Ye, Zihao Sheng.

**Figure 2.** Figure 2: Model architecture and training paradigm of ExploreVLA. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Figure 3: Analysis of the exploration bonus. Left: the exploration bonus is positively correlated with L2 error to the ground-truth trajectory. Right: our exploration bonus can properly measure the trajectory novelty that L2 error fails. 4.3 Analysis of Intrinsic Reward Modeling [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

**Figure 4.** Figure 4: Qualitative comparison of planned trajectories before and after RL [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗

**Figure 5.** Figure 5: Additional qualitative results on the navtest split. We visualize the planned trajectories across three scenario categories: Going Straight, Turning, and Intersection. For each example, we show the front-view camera image and the corresponding BEV representation with trajectories overlaid (green: GT, orange: prediction) [PITH_FULL_IMAGE:figures/full_fig_p020_5.png] view at source ↗

**Figure 6.** Figure 6: Qualitative results of dense world modeling on the [PITH_FULL_IMAGE:figures/full_fig_p021_6.png] view at source ↗

discussion (0)

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Driving
cs.CV 2026-05 unverdicted novelty 6.0

CoPhy distills VLM knowledge into a BEV encoder and uses an action-conditioned auto-regressive BEV world model inside GRPO with dual physical-cognitive rewards to reach SOTA on NAVSIM v1/v2 while adding language-based...
Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Driving
cs.CV 2026-05 unverdicted novelty 5.0

CoPhy is a new RL framework that distills VLM cognition into BEV encoders, adds an auto-regressive BEV world model for action-conditioned future prediction, and optimizes policies via GRPO with dual physical-cognitive...