Finite-Time Analysis of Asynchronous Stochastic Approximation and $Q$-Learning

Adam Wierman; Guannan Qu

Finite-Time Analysis of Asynchronous Stochastic Approximation and Q-Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2002.00260 v1 pith:6LQS5TMM submitted 2020-02-01 math.OC cs.LG

Finite-Time Analysis of Asynchronous Stochastic Approximation and Q-Learning

Guannan Qu , Adam Wierman This is my paper

classification math.OC cs.LG

keywords asynchronouslearningboundapproximationfinite-timestochasticadditionallyanalysis

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

0 comments

read the original abstract

We consider a general asynchronous Stochastic Approximation (SA) scheme featuring a weighted infinity-norm contractive operator, and prove a bound on its finite-time convergence rate on a single trajectory. Additionally, we specialize the result to asynchronous $Q$-learning. The resulting bound matches the sharpest available bound for synchronous $Q$-learning, and improves over previous known bounds for asynchronous $Q$-learning.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Toward a Unified Lyapunov-Certified ODE Convergence Analysis of Smooth Q-Learning with p-Norms
cs.LG 2024-04 unverdicted novelty 5.0

Unified ODE convergence analysis for smooth Q-learning variants via p-norm Lyapunov functions, valid even when the Bellman operator is not a contraction.