Neural Contextual Bandits with Deep Representation and Shallow Exploration
read the original abstract
We study a general class of contextual bandits, where each context-action pair is associated with a raw feature vector, but the reward generating function is unknown. We propose a novel learning algorithm that transforms the raw feature vector using the last hidden layer of a deep ReLU neural network (deep representation learning), and uses an upper confidence bound (UCB) approach to explore in the last linear layer (shallow exploration). We prove that under standard assumptions, our proposed algorithm achieves $\tilde{O}(\sqrt{T})$ finite-time regret, where $T$ is the learning time horizon. Compared with existing neural contextual bandit algorithms, our approach is computationally much more efficient since it only needs to explore in the last layer of the deep neural network.
This paper has not been read by Pith yet.
Forward citations
Cited by 4 Pith papers
-
Neural Variance-aware Dueling Bandits with Deep Representation and Shallow Exploration
Variance-aware neural dueling bandit algorithms achieve sublinear regret of order O(d sqrt(sum sigma_t^2) + sqrt(d T)) for wide networks on nonlinear utilities.
-
Neural Exploitation and Exploration of Contextual Bandits
EE-Net is a contextual bandit algorithm that pairs an exploitation neural net with a separate exploration neural net and proves an instance-dependent Õ(√T) regret bound while beating linear and neural baselines on real data.
-
Balancing Expressivity and Learnability in Quantum Kernel Bandit Optimization
Proposes projected quantum kernels with misspecified GP bandit algorithms and regret bounds to trade off expressivity against learnability in quantum kernel optimization.
-
Variational Proximal Policy Optimization
VP2O maps PPO to SVGD in a MoE architecture using functional kernels and expert orthogonalization, claiming +179 ELO on Codeforces and 32% token reduction on AIME for a 33B/4B model.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.