ENAP extracts an emergent Mealy automaton from visuomotor trajectories to act as a high-level planner for a low-level residual policy, yielding up to 27% higher success than end-to-end VLA policies in low-data regimes.
Learning Finite State Representations of Recurrent Policy Networks
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
Recurrent neural networks (RNNs) are an effective representation of control policies for a wide range of reinforcement and imitation learning problems. RNN policies, however, are particularly difficult to explain, understand, and analyze due to their use of continuous-valued memory vectors and observation features. In this paper, we introduce a new technique, Quantized Bottleneck Insertion, to learn finite representations of these vectors and features. The result is a quantized representation of the RNN that can be analyzed to improve our understanding of memory use and general behavior. We present results of this approach on synthetic environments and six Atari games. The resulting finite representations are surprisingly small in some cases, using as few as 3 discrete memory states and 10 observations for a perfect Pong policy. We also show that these finite policy representations lead to improved interpretability.
verdicts
UNVERDICTED 2representative citing papers
A translation method converts finite-state-controller policies for POMDPs into a decision-tree-plus-Mealy-machine form that is typically smaller and more explainable, with further simplifications for attractor-based policies.
citing papers explorer
-
Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories
ENAP extracts an emergent Mealy automaton from visuomotor trajectories to act as a high-level planner for a low-level residual policy, yielding up to 27% higher success than end-to-end VLA policies in low-data regimes.
-
Explainable Representation of Finite-Memory Policies for POMDPs using Decision Trees
A translation method converts finite-state-controller policies for POMDPs into a decision-tree-plus-Mealy-machine form that is typically smaller and more explainable, with further simplifications for attractor-based policies.