REVIEW 13 cited by
Learning Invariant Representations for Reinforcement Learning without Reconstruction
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Learning Invariant Representations for Reinforcement Learning without Reconstruction
read the original abstract
We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction. Our goal is to learn representations that both provide for effective downstream control and invariance to task-irrelevant details. Bisimulation metrics quantify behavioral similarity between states in continuous MDPs, which we propose using to learn robust latent representations which encode only the task-relevant information from observations. Our method trains encoders such that distances in latent space equal bisimulation distances in state space. We demonstrate the effectiveness of our method at disregarding task-irrelevant information using modified visual MuJoCo tasks, where the background is replaced with moving distractors and natural videos, while achieving SOTA performance. We also test a first-person highway driving task where our method learns invariance to clouds, weather, and time of day. Finally, we provide generalization results drawn from properties of bisimulation metrics, and links to causal inference.
Forward citations
Cited by 13 Pith papers
-
Textual Belief States for World Models: Identifiable Representation Learning Under Strict Mediation
Introduces textual belief states and factorized GRPO to enforce strict latent state mediation in text-based world models, yielding preserved prediction accuracy with large gains in representation quality and rollout p...
-
BiPACE: Bisimulation-Guided Policy Optimization with Action Counterfactual Estimation for LLM Agents
BiPACE improves LLM agent policy optimization by using bisimulation proxies from hidden states for step clustering and action-conditioned baselines for advantage estimation, raising success rates on ALFWorld, WebShop,...
-
Beyond Next-Observation Prediction: Agent-Authored World Modeling for Sequential Decision Making
AAWM builds training targets for world models by retrieving and synthesizing transition evidence based on the policy's self-identified decision needs at each state.
-
Learning Object Manipulation from Scratch via Contrastive Interaction
IWR improves CRL sample efficiency and performance in interaction-rich manipulation by interaction-aware resampling that preserves mode boundaries, yielding 19.8% average gains and a real-world air-hockey agent.
-
Topology-Aware State Abstraction with Tangle Cores for Markov Decision Processes
Tangle-core abstraction creates overlapping state abstractions for MDPs from graph tangles, with value-preservation guarantees under action consistency and improved empirical compression-return tradeoffs in bottleneck...
-
Matrix-Space Reinforcement Learning for Reusing Local Transition Geometry
MSRL represents trajectory segments as PSD matrices to prove additive composition properties and bootstrap value functions for better transfer, reaching 0.73 AUC versus 0.57-0.65 baselines.
-
What to Ignore, What to React: Visually Robust RL Fine-Tuning of VLA Models
PAIR-VLA adds invariance and sensitivity objectives over paired visual variants during PPO fine-tuning of VLA models, yielding 9-16% average gains on ManiSkill3 under distractors, textures, poses, viewpoints, and ligh...
-
Predictive but Not Plannable: RC-aux for Latent World Models
RC-aux corrects spatiotemporal mismatch in reconstruction-free latent world models by adding multi-horizon prediction and reachability supervision, improving planning performance on goal-conditioned pixel-control tasks.
-
Next-Latent Prediction Transformers Learn Compact World Models
NextLat augments next-token prediction with latent next-state prediction, theoretically converging latents to belief states and showing empirical gains in world modeling, reasoning, planning, and faster inference via ...
-
Bayesian Inverse Transition Learning: Learning Dynamics From Near-Optimal Trajectories
A Bayesian method uses near-optimality constraints from expert trajectories to estimate transition dynamics in offline model-based reinforcement learning.
-
R3M: A Universal Visual Representation for Robot Manipulation
A visual encoder pre-trained on diverse human videos with contrastive and language objectives improves simulated robot manipulation success by over 20% versus training from scratch and enables real Franka arm tasks fr...
-
Direct Advantage Estimation for Scalable and Sample-efficient Deep Reinforcement Learning
Extends DAE theory to POMDPs with minimal changes and introduces discrete latent dynamics to cut computational cost, with ALE experiments showing scalability and retained sample efficiency.
-
Optimal Control with Natural Images: Efficient Reinforcement Learning using Overcomplete Sparse Codes
Overcomplete sparse coding of natural images enables reinforcement learning to solve optimal control tasks orders of magnitude larger than with complete codes, via a new scalable benchmark and theoretical justification.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.