REVIEW 36 cited by
Policy Distillation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Policy Distillation
read the original abstract
Policies for complex visual tasks have been successfully learned with deep reinforcement learning, using an approach called deep Q-networks (DQN), but relatively large (task-specific) networks and extensive training are needed to achieve good performance. In this work, we present a novel method called policy distillation that can be used to extract the policy of a reinforcement learning agent and train a new network that performs at the expert level while being dramatically smaller and more efficient. Furthermore, the same method can be used to consolidate multiple task-specific policies into a single policy. We demonstrate these claims using the Atari domain and show that the multi-task distilled agent outperforms the single-task teachers as well as a jointly-trained DQN agent.
Forward citations
Cited by 36 Pith papers
-
Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedback
Noisy expert imitation learning requires exponential samples for offline methods but polynomial for a variant of on-policy distillation under a noise condition.
-
World Model Self-Distillation: Training World Models to Solve General Tasks
Self-distillation from a caption-conditioned video diffusion model to an image-and-prompt-conditioned executor, enhanced by RL from VLM feedback, enables task solving in world models.
-
FedQHD: Closed-Form Function-Space Federated Reinforcement Learning
FedQHD achieves closed-form federated Q-learning via hyperdimensional encoders with linear readouts, formalizes the federation gap under heterogeneous encoders, and reports competitive performance on continuous-state ...
-
What Does Deep Hedging Actually Learn? Delta Corrections, Regime Fragility, and Symbolic Distillation
Deep hedging agents learn a systematic delta haircut explained by spot-implied-volatility co-movement; symbolic regression distills the policies into formulas that retain reward and downside-variance advantages over B...
-
VISD: Enhancing Video Reasoning via Structured Self-Distillation
VISD improves VideoLLM reasoning performance and training efficiency by combining structured multi-dimensional self-distillation feedback with RL via direction-magnitude decoupling, curriculum scheduling, and EMA stab...
-
SafeAdapt: Provably Safe Policy Updates in Deep Reinforcement Learning
SafeAdapt certifies a Rashomon set of safe policies from demonstration data and projects updates from arbitrary RL algorithms onto it to guarantee preservation of safety on source tasks.
-
Demystifying OPD: Length Inflation and Stabilization Strategies for Large Language Models
OPD for LLMs suffers length inflation and repetition collapse; StableOPD uses reference divergence and rollout mixing to prevent it and improve math reasoning performance by 7.2% on average.
-
ROAD-VLA: Robust Online Adaptation via Self-Distillation for Vision-Language-Action Models
ROAD-VLA constructs an advantage-perturbed proximal teacher in action space to convert sparse rewards into dense supervision for online VLA adaptation and reports outperformance versus PPO across seven manipulation en...
-
MotionPyramid: Hierarchical Motion Representation and Residual Interfaces
MotionPyramid learns a stack of latent decoders from motion tracking data to create multi-resolution action interfaces for RL policies in humanoid control, with residual interfaces allowing coarse programs and fine co...
-
Dynamics Are Learned, Not Told: Semi-Supervised Discovery of Latent Dynamics Geometries For Zero-Shot Policy Adaptation
Contrastive learning bounds the Lipschitz constant of a trajectory dynamics encoder to support outcome-centric zero-shot adaptation in MuJoCo robotics tasks under severe dynamics shifts.
-
De-attribute to Forget for LLM Unlearning
DareU reframes LLM unlearning as zeroing data attribution via RL rewards from an LLM classifier approximation, claiming better balance of forget quality and model utility than loss-based baselines.
-
Critic-Driven Voronoi-Quantization for Distilling Deep RL Policies to Explainable Models
Critic-Driven Voronoi State Partitioning distills deep RL policies into piecewise-linear models by iteratively adding linear subpolicies in high-value-error regions identified by the critic.
-
Policy-DRIFT: Dynamic Reward-Informed Flow Trajectory Steering
Policy-DRIFT combines conditional flow matching with terminal reward guidance and decoupled DRL to achieve 49% drag reduction in Re_tau=180 channel flow, 16% above DRL benchmarks and with 37 times less actuation energy.
-
Demystifying Deep Reinforcement Learning: A Neuro-Symbolic Framework for Interpretable Open RAN Automation
DeRAN converts black-box DRL policies into interpretable symbolic representations for O-RAN automation, retaining 78-87% of original performance while adding built-in transparency.
-
Demystifying Deep Reinforcement Learning: A Neuro-Symbolic Framework for Interpretable Open RAN Automation
DeRAN converts opaque DRL policies for O-RAN tasks into interpretable symbolic policies via concept abstraction, deep symbolic regression, and neurally guided logic, retaining 78-87% of DRL performance on a live 5G testbed.
-
Precise Aggressive Aerial Maneuvers with Sensorimotor Policies
Reinforcement learning sensorimotor policies enable quadrotors to traverse narrow gaps at extreme tilts with 5 cm clearance using only vision and proprioception, including reactive traversal of moving gaps.
-
Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
Process advantage verifiers trained to predict step-level progress under a distinct prover policy improve LLM reasoning accuracy by over 8% and sample efficiency by 5-6x over outcome reward models.
-
Continual Domain Randomization
Continual Domain Randomization trains RL policies sequentially on randomization parameter subsets with continual learning to achieve robust sim-to-real transfer in robotic reaching and grasping.
-
MiniLLM: On-Policy Distillation of Large Language Models
MiniLLM distills large language models into smaller ones via reverse KL divergence and on-policy optimization, yielding higher-quality responses with lower exposure bias than standard KD baselines.
-
Attentive Multi-Task Deep Reinforcement Learning
Attention mechanism dynamically groups task knowledge at state granularity in multi-task DRL to enable positive transfer and avoid negative transfer, matching or exceeding prior methods with fewer parameters.
-
Progressive Neural Networks
Progressive neural networks learn sequences of RL tasks without catastrophic forgetting by freezing prior columns and adding lateral connections for knowledge transfer.
-
UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillation
UCOB improves agentic RL by using return-to-go comparisons between skill-conditioned and no-skill prompts as local teachers for bidirectional self-distillation and skill memory updates.
-
Regularized Reward-Punishment Reinforcement Learning
Introduces KCPR and its deep form klDMP that couples reward and punishment policies via learned priors, yielding improved safety and stability in grid-world and Gazebo navigation tasks over DQN, SQL and softDMP.
-
Belief-Aware Scheduling for Predictive Wildfire Hazard Mapping under Sparse-Window Telemetry
The paper shows that deriving a structured belief from the prediction operator's needs and using it in non-myopic scheduling yields up to 28% better predictive loss than activity-paced baselines on a physics-calibrate...
-
Robot Squid Game: Quadrupedal Locomotion for Traversing Narrow Tunnels
A teacher-student RL policy distillation approach combined with procedural tunnel generation enables quadruped robots to traverse narrow tunnels consistently in both simulation and real-world tests.
-
VISD: Enhancing Video Reasoning via Structured Self-Distillation
VISD improves VideoLLM reasoning by adding multi-dimensional diagnostic self-distillation and RL decoupling, yielding higher accuracy, better grounding, and nearly 2x faster training convergence.
-
VISD: Enhancing Video Reasoning via Structured Self-Distillation
VISD proposes structured self-distillation with a multi-dimensional judge model and direction-magnitude decoupling to improve token-level credit assignment and convergence speed in VideoLLM reasoning training.
-
VISD: Enhancing Video Reasoning via Structured Self-Distillation
VISD adds structured privileged feedback from a judge model and a direction-magnitude decoupling trick to let VideoLLMs learn token-level credit assignment while keeping RL stable, yielding higher accuracy and roughly...
-
LANTERN: LLM-Augmented Neurosymbolic Transfer with Experience-Gated Reasoning Networks
LANTERN improves RL sample efficiency by 40-60% via LLM-generated task automata, semantic multi-source policy aggregation, and experience-gated adaptive transfer.
-
Combining Trained Models in Reinforcement Learning
A review of 15 studies finds positive transfer in DRL mainly when source and target tasks share structure or include alignment mechanisms, but compute-matched comparisons against from-scratch baselines remain rare.
-
Detecting is Easy, Adapting is Hard: Local Expert Growth for Visual Model-Based Reinforcement Learning under Distribution Shift
JEPA-Indexed Local Expert Growth adds local action corrections for detected shift clusters and yields statistically significant OOD gains on four shift conditions while keeping in-distribution performance intact.
-
ReFineVLA: Multimodal Reasoning-Aware Generalist Robotic Policies via Teacher-Guided Fine-Tuning
ReFineVLA adds teacher-generated reasoning steps to VLA training and reports state-of-the-art success rates on SimplerEnv WidowX and Google Robot benchmarks.
-
Sparse Sensor Placement in Multi-Agent Reinforcement Learning Control of Rayleigh-B\'enard Convection
Distills sparse multi-agent RL policies for Rayleigh-Bénard convection control via grouped regularization, achieving high sparsity while retaining performance comparable to dense experts.
-
GCT-MARL: Graph-Based Contrastive Transfer for Sample-Efficient Cooperative Multi-Agent Reinforcement Learning
GCT-MARL augments a multi-view graph contrastive backbone with per-view adaptive alignment loss and two-phase training to accelerate convergence in cooperative MARL transfer across homogeneous and heterogeneous agent ...
-
PRIDE: Privileged Information-enhanced Distillation for Empathetic Dialogue Generation
PRIDE distills empathetic reasoning from large teacher LLMs to smaller students via an empathy prompt, multi-source attention, and dual-alignment loss using privileged information available only at training time.
-
Digital Guardians: The Past and The Future of Cyber-Physical Resilience
A survey frames CPS resilience through five themes and illustrates them in connected transportation and medical systems to provide a roadmap for real-world resilience.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.