pith. sign in

arxiv: 1802.03006 · v1 · pith:CJOGZAZJnew · submitted 2018-02-08 · 💻 cs.LG

Learning and Querying Fast Generative Models for Reinforcement Learning

Lars Buesing , Theophane Weber , Sebastien Racaniere , S. M. Ali Eslami , Danilo Rezende , David P. Reichert , Fabio Viola , Frederic Besse
show 3 more authors
Karol Gregor Demis Hassabis Daan Wierstra
This is my paper
classification 💻 cs.LG
keywords modelslearningenvironmentstate-spacecomputationalgenerativereinforcementaccuracy
0
0 comments X
read the original abstract

A key challenge in model-based reinforcement learning (RL) is to synthesize computationally efficient and accurate environment models. We show that carefully designed generative models that learn and operate on compact state representations, so-called state-space models, substantially reduce the computational costs for predicting outcomes of sequences of actions. Extensive experiments establish that state-space models accurately capture the dynamics of Atari games from the Arcade Learning Environment from raw pixels. The computational speed-up of state-space models while maintaining high accuracy makes their application in RL feasible: We demonstrate that agents which query these models for decision making outperform strong model-free baselines on the game MSPACMAN, demonstrating the potential of using learned environment models for planning.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

    cs.LG 2019-11 accept novelty 8.0

    MuZero matches or exceeds AlphaZero-level performance in Go, Chess, Shogi and sets a new state of the art on 57 Atari games by learning a model that directly supports planning rather than reconstructing full environme...

  2. Learning to Theorize the World from Observation

    cs.LG 2026-05 unverdicted novelty 7.0

    NEO is a probabilistic neural model that induces compositional programs as a learned Language of Thought from non-textual observations and executes them via a shared transition model to enable explanation-driven gener...

  3. Mastering Atari with Discrete World Models

    cs.LG 2020-10 accept novelty 7.0

    DreamerV2 reaches human-level performance on 55 Atari games by learning behaviors inside a separately trained discrete-latent world model.

  4. Dream to Control: Learning Behaviors by Latent Imagination

    cs.LG 2019-12 accept novelty 7.0

    Dreamer learns to control from images by imagining and optimizing behaviors in a learned latent world model, outperforming prior methods on 20 visual tasks in data efficiency and final performance.

  5. Learning to Theorize the World from Observation

    cs.LG 2026-05 unverdicted novelty 6.0

    NEO induces compositional latent programs as world theories from observations and executes them to enable explanation-driven generalization.

  6. Learning Belief Representations for Imitation Learning in POMDPs

    cs.LG 2019-06 unverdicted novelty 6.0

    BMIL learns belief modules jointly with policies for GAIL-style imitation learning in POMDPs, outperforming separate training and standard GAIL on continuous control tasks.

  7. Shaping Belief States with Generative Environment Models for RL

    cs.LG 2019-06 unverdicted novelty 5.0

    Multi-step predictive generative models form stable belief states capturing environment layout and agent pose, yielding higher data efficiency on RL tasks than model-free agents.