Pith. sign in

REVIEW 2 minor 2 cited by

Online nonstochastic control targets low regret against the best hindsight policy when both costs and dynamics are chosen by an adversary.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-05-24 11:15 UTC

load-bearing objection This is a straightforward expository introduction to online nonstochastic control that restates the OCO reduction without new results or proofs.

arxiv 2211.09619 v8 submitted 2022-11-17 cs.LG cs.ROcs.SYeess.SYmath.OCstat.ML

Introduction to Online Control

Elad Hazan , Karan Singh This is my paper
classification cs.LG cs.ROcs.SYeess.SYmath.OCstat.ML
keywords online controlnonstochastic controlonline convex optimizationregret boundsdynamical systemsadversarial perturbationsrobust controlreinforcement learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces online nonstochastic control as a framework for dynamical systems in which cost functions and perturbations from the assumed model are selected adversarially. The objective is no longer to match an offline optimal strategy under stochastic assumptions, but instead to achieve low regret relative to the best policy from a benchmark class evaluated in hindsight. The approach applies techniques from online convex optimization and convex relaxations to derive iterative algorithms that come with finite-time regret bounds and computational complexity guarantees. A reader would care because the framework supplies performance assurances in settings where no fixed optimal policy exists in advance.

Core claim

The paper presents online nonstochastic control as a paradigm that uses online convex optimization on convex relaxations of control problems to obtain methods with provable regret and complexity guarantees. In this setting the optimal policy is not defined a priori because both the cost functions and the perturbations from the assumed dynamical model are chosen by an adversary; the target is therefore low regret against the best policy in hindsight from a benchmark class of policies.

What carries the argument

Application of online convex optimization and convex relaxations to produce iterative algorithms that minimize regret in adversarial control settings.

Load-bearing premise

Both the cost functions and the perturbations are chosen by an adversary so that the optimal policy is not fixed in advance and the goal becomes regret against the best hindsight policy.

What would settle it

A concrete counter-example in which the derived iterative algorithms fail to achieve sublinear regret against the best benchmark policy on a simple linear dynamical system with adversarial costs and perturbations.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Iterative optimization algorithms yield finite-time regret bounds for classical control settings under adversarial conditions.
  • Computational complexity guarantees accompany the regret bounds for the resulting methods.
  • The framework applies to both optimal control and robust control problems once the objective is changed to hindsight regret minimization.
  • No fixed optimal policy needs to be known or computed upfront for the guarantees to hold.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same regret-minimization approach could be tested in reinforcement learning environments that switch between different linear dynamics without prior notice.
  • It may be possible to derive explicit regret bounds for policy classes that include nonlinear or time-varying controllers by extending the convex relaxation step.
  • The distinction between stochastic and adversarial noise suggests new benchmark problems that separate the two regimes in simulation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 2 minor

Summary. The manuscript is an expository introduction to online nonstochastic control. It distinguishes this paradigm from optimal and robust control by noting that both cost functions and dynamical perturbations are chosen adversarially, so that the goal is low regret relative to the best policy in hindsight from a benchmark class rather than performance matching an a priori optimal policy. The text advocates importing techniques from online convex optimization, resulting in methods based on iterative convex optimization that come with finite-time regret and computational complexity guarantees.

Significance. If the exposition is accurate and self-contained, the manuscript could provide a useful entry point for control and RL researchers into the online nonstochastic control framework and its reduction to OCO. No machine-checked proofs, reproducible code, or novel falsifiable predictions are claimed; the value is therefore primarily pedagogical rather than technical.

minor comments (2)
  1. [Abstract] Abstract, paragraph 3: the statement that the methods 'are accompanied by finite-time regret and computational complexity guarantees' is presented without any derivation, citation to a specific theorem, or pointer to the relevant OCO reduction; for an introductory text this is acceptable only if the body supplies the missing references or sketches.
  2. The manuscript should include at least one concrete low-dimensional example (e.g., scalar linear system with quadratic costs) that illustrates how the OCO reduction is instantiated and what the benchmark policy class looks like.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive assessment of the manuscript as a pedagogical introduction to online nonstochastic control and for recommending minor revision. No specific major comments are provided in the report, so there are no individual points requiring point-by-point rebuttal. We will incorporate any minor suggestions during revision to ensure the exposition remains accurate and self-contained.

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper is an expository introduction that imports standard online convex optimization (OCO) techniques to the control setting. The central objective (low regret vs. best-in-hindsight policy from a benchmark class) is defined explicitly, and the methods are described as reductions to iterative convex optimization with known regret guarantees. No derivation chain, fitted parameter, or self-citation is used to establish the core claims; the argument structure is self-contained against external OCO benchmarks and does not reduce any result to its own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Based solely on the abstract, the framework rests on standard domain assumptions from online convex optimization and control theory; no free parameters, invented entities, or ad-hoc axioms are introduced in the provided text.

axioms (1)
  • domain assumption Cost functions and convex relaxations are convex
    The application of online convex optimization requires convexity of the costs and relaxations used in the control setting.

pith-pipeline@v0.9.0 · 5712 in / 1125 out tokens · 33201 ms · 2026-05-24T11:15:33.066158+00:00 · methodology

0 comments
read the original abstract

This text presents an introduction to an emerging paradigm in control of dynamical systems and differentiable reinforcement learning called online nonstochastic control. The new approach applies techniques from online convex optimization and convex relaxations to obtain new methods with provable guarantees for classical settings in optimal and robust control. The primary distinction between online nonstochastic control and other frameworks is the objective. In optimal control, robust control, and other control methodologies that assume stochastic noise, the goal is to perform comparably to an offline optimal strategy. In online nonstochastic control, both the cost functions as well as the perturbations from the assumed dynamical model are chosen by an adversary. Thus the optimal policy is not defined a priori. Rather, the target is to attain low regret against the best policy in hindsight from a benchmark class of policies. This objective suggests the use of the decision making framework of online convex optimization as an algorithmic methodology. The resulting methods are based on iterative mathematical optimization algorithms, and are accompanied by finite-time regret and computational complexity guarantees.

Figures

Figures reproduced from arXiv: 2211.09619 by Elad Hazan, Karan Singh.

Figure 1.1
Figure 1.1. Figure 1.1: A centrifugal governor. Source: Wikipedia [PITH_FULL_IMAGE:figures/full_fig_p016_1_1.png] view at source ↗
Figure 1.2
Figure 1.2. Figure 1.2: Performance of the PID controller on a mechanical ventilator [PITH_FULL_IMAGE:figures/full_fig_p019_1_2.png] view at source ↗
Figure 2.1
Figure 2.1. Figure 2.1: A schematic of the respiratory circuit from [SZG [PITH_FULL_IMAGE:figures/full_fig_p028_2_1.png] view at source ↗
Figure 2.2
Figure 2.2. Figure 2.2: The position and velocity visualized for a double integrator. [PITH_FULL_IMAGE:figures/full_fig_p029_2_2.png] view at source ↗
Figure 2.3
Figure 2.3. Figure 2.3: A pivoted massless rod connected to a point object with mass [PITH_FULL_IMAGE:figures/full_fig_p030_2_3.png] view at source ↗
Figure 2.4
Figure 2.4. Figure 2.4: Longitudinal control in an aircraft. Source: Wikipedia [PITH_FULL_IMAGE:figures/full_fig_p031_2_4.png] view at source ↗
Figure 2.5
Figure 2.5. Figure 2.5: Trajectory of a continuous-time SIR model. Source: Wikipedia [PITH_FULL_IMAGE:figures/full_fig_p032_2_5.png] view at source ↗
Figure 2.6
Figure 2.6. Figure 2.6: Julia set [PITH_FULL_IMAGE:figures/full_fig_p035_2_6.png] view at source ↗
Figure 3.1
Figure 3.1. Figure 3.1: Source: Reinforcement Learning: An Introduction. Sutton & [PITH_FULL_IMAGE:figures/full_fig_p040_3_1.png] view at source ↗
Figure 3.2
Figure 3.2. Figure 3.2: Source: [RN02]. In this example, the robot has controls that allow it to move left / right / forward / backward. The controls, however, are not reliable, and with 0.8 probably the robot moves to wherever it intends. With the remaining 0.2 probability the robot moves sideways, 0.1 probability to each side. Notice that the Markovian assumption clearly holds for this simple ex￾ample. What is an optimal solu… view at source ↗
Figure 3.3
Figure 3.3. Figure 3.3: The optimal policy (mapping from state to action) with reward [PITH_FULL_IMAGE:figures/full_fig_p041_3_3.png] view at source ↗
Figure 3.4
Figure 3.4. Figure 3.4: Here, we change the reward of each step to be -2. The optimal [PITH_FULL_IMAGE:figures/full_fig_p042_3_4.png] view at source ↗
Figure 3.5
Figure 3.5. Figure 3.5: In this very simple Markov Chain, the state alternates every [PITH_FULL_IMAGE:figures/full_fig_p044_3_5.png] view at source ↗
Figure 3.6
Figure 3.6. Figure 3.6: Adding a self-loop to every node makes the periodicity to be [PITH_FULL_IMAGE:figures/full_fig_p044_3_6.png] view at source ↗
Figure 4.1
Figure 4.1. Figure 4.1: Four different deterministic linear time-invariant dynamical sys [PITH_FULL_IMAGE:figures/full_fig_p055_4_1.png] view at source ↗
Figure 6
Figure 6. Figure 6: for fully observed systems, and in Figure 8.1 for partially observed [PITH_FULL_IMAGE:figures/full_fig_p080_6.png] view at source ↗
Figure 6.1
Figure 6.1. Figure 6.1: Schematic relationship between linear Disturbance-Action Con [PITH_FULL_IMAGE:figures/full_fig_p081_6_1.png] view at source ↗
Figure 8.1
Figure 8.1. Figure 8.1: Schematic relationship between linear Disturbance-Response [PITH_FULL_IMAGE:figures/full_fig_p102_8_1.png] view at source ↗
Figure 11.1
Figure 11.1. Figure 11.1: In learning we passively observe the input sequence and attempt [PITH_FULL_IMAGE:figures/full_fig_p127_11_1.png] view at source ↗
Figure 13.1
Figure 13.1. Figure 13.1: Eigenvalues of Hankel matrices decrease geometrically. These [PITH_FULL_IMAGE:figures/full_fig_p146_13_1.png] view at source ↗
Figure 13.2
Figure 13.2. Figure 13.2: The filters obtained by the eigenvectors of [PITH_FULL_IMAGE:figures/full_fig_p150_13_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Online Nonstochastic Prediction: Logarithmic Regret via Predictive Online Least Squares

    cs.LG 2026-05 unverdicted novelty 7.0

    Predictive hints from any stabilizing Luenberger observer make hint residuals uniformly bounded in online least squares, yielding logarithmic regret for nonstochastic prediction despite unbounded trajectories in margi...

  2. Steady-state Based Approach to Online Non-stochastic Control

    math.OC 2026-04 unverdicted novelty 6.0

    A new online algorithm for adversarial linear control achieves square-root regret against steady-states attainable under affine controllers rather than constant inputs.

Reference graph

Works this paper leans on

100 extracted references · 100 canonical work pages · cited by 2 Pith papers · 3 internal anchors

  1. [1]

    Online control with adversarial disturbances

    Naman Agarwal, Brian Bullins, Elad Hazan, Sham Kakade, and Karan Singh. Online control with adversarial disturbances. In International Conference on Machine Learning , pages 111--119. PMLR, 2019

    work page 2019

  2. [2]

    Special report: The simulations driving the world's response to covid-19

    David Adam. Special report: The simulations driving the world's response to covid-19. Nature , 580(7802):316--319, 2020

    work page 2020

  3. [3]

    System level synthesis

    James Anderson, John C Doyle, Steven H Low, and Nikolai Matni. System level synthesis. Annual Reviews in Control , 47:364--393, 2019

    work page 2019

  4. [4]

    Online Bandit Learning against an Adaptive Adversary: from Regret to Policy Regret

    Raman Arora, Ofer Dekel, and Ambuj Tewari. Online bandit learning against an adaptive adversary: from regret to policy regret. arXiv preprint arXiv:1206.6400 , 2012

    work page internal anchor Pith review Pith/arXiv arXiv 2012

  5. [5]

    o m and Tore H \

    Karl Johan str \"o m and Tore H \"a gglund. PID controllers: theory, design, and tuning , volume 2. Instrument society of America Research Triangle Park, NC, 1995

    work page 1995

  6. [6]

    On the difficulty of deciding asymptotic stability of cubic homogeneous vector fields

    Amir Ali Ahmadi. On the difficulty of deciding asymptotic stability of cubic homogeneous vector fields. In 2012 American Control Conference (ACC) , pages 3334--3339. IEEE, 2012

    work page 2012

  7. [7]

    Online Convex Optimization Against Adversaries with Memory and Application to Statistical Arbitrage

    Oren Anava, Elad Hazan, and Shie Mannor. Online convex optimization against adversaries with memory and application to statistical arbitrage. arXiv preprint arXiv:1302.6937 , 2013

    work page internal anchor Pith review Pith/arXiv arXiv 2013

  8. [8]

    ORF 363: Computing and Optimization

    Amir Ali Ahmadi. ORF 363: Computing and Optimization . http://aaa.princeton.edu/orf363, 2016. [Online; accessed August 1, 2022]

    work page 2016

  9. [9]

    A regret minimization approach to iterative learning control

    Naman Agarwal, Elad Hazan, Anirudha Majumdar, and Karan Singh. A regret minimization approach to iterative learning control. In International Conference on Machine Learning , pages 100--109. PMLR, 2021

    work page 2021

  10. [10]

    Logarithmic regret for online control

    Naman Agarwal, Elad Hazan, and Karan Singh. Logarithmic regret for online control. In Advances in Neural Information Processing Systems , pages 10175--10184, 2019

    work page 2019

  11. [11]

    Lectures on the theory of reinforcement learning

    Alekh Agarwal, Nan Jiang, Sham M Kakade, and Wen Sun. Lectures on the theory of reinforcement learning. 2019

    work page 2019

  12. [12]

    Kalman filtering: Whence, what and whither? Mathematical System Theory: The Influence of RE Kalman , pages 41--54, 1991

    BDO Anderson and JB Moore. Kalman filtering: Whence, what and whither? Mathematical System Theory: The Influence of RE Kalman , pages 41--54, 1991

    work page 1991

  13. [13]

    personal communications

    Sanjeev Arora. personal communications. 2020

    work page 2020

  14. [14]

    Tracking adversarial targets

    Yasin Abbasi-Yadkori, Peter Bartlett, and Varun Kanade. Tracking adversarial targets. In International Conference on Machine Learning , pages 369--377. PMLR, 2014

    work page 2014

  15. [15]

    Regret bounds for the adaptive control of linear quadratic systems

    Yasin Abbasi-Yadkori and Csaba Szepesv \'a ri. Regret bounds for the adaptive control of linear quadratic systems. In Proceedings of the 24th Annual Conference on Learning Theory , pages 1--26, 2011

    work page 2011

  16. [16]

    H-infinity optimal control and related minimax design problems: a dynamic game approach

    Tamer Ba s ar and Pierre Bernhard. H-infinity optimal control and related minimax design problems: a dynamic game approach . Springer Science & Business Media, 2008

    work page 2008

  17. [17]

    S. Bennett . Development of the pid controller. IEEE Control Systems Magazine , 13(6):58--62, 1993

    work page 1993

  18. [18]

    Dynamic Programming and Optimal Control

    Dimitri P Bertsekas. Dynamic Programming and Optimal Control . Athena Scientific, 2007

    work page 2007

  19. [19]

    Reinforcement learning and optimal control

    Dimitri Bertsekas. Reinforcement learning and optimal control . Athena Scientific, 2019

    work page 2019

  20. [20]

    Np-hardness of some linear control design problems

    Vincent Blondel and John N Tsitsiklis. Np-hardness of some linear control design problems. SIAM journal on control and optimization , 35(6):2118--2127, 1997

    work page 1997

  21. [21]

    Three problems on the decidability and complexity of stability

    Vincent D Blondel and John N Tsitsiklis. Three problems on the decidability and complexity of stability. In Open Problems in Mathematical Systems and Control Theory , pages 45--51. Springer, 1999

    work page 1999

  22. [22]

    A survey of computational complexity results in systems and control

    Vincent D Blondel and John N Tsitsiklis. A survey of computational complexity results in systems and control. Automatica , 36(9):1249--1274, 2000

    work page 2000

  23. [23]

    On the singular values of matrices with displacement structure

    Bernhard Beckermann and Alex Townsend. On the singular values of matrices with displacement structure. SIAM Journal on Matrix Analysis and Applications , 38(4):1227--1248, 2017

    work page 2017

  24. [24]

    Computability of Julia sets

    Mark Braverman and Michael Yampolsky. Computability of Julia sets . Springer, 2008

    work page 2008

  25. [25]

    Rate-optimal online convex optimization in adaptive linear control

    Asaf Cassel, Alon Cohen, and Tomer Koren. Rate-optimal online convex optimization in adaptive linear control. arXiv preprint arXiv:2206.01426 , 2022

    work page arXiv 2022

  26. [26]

    Black-box control for linear dynamical systems

    Xinyi Chen and Elad Hazan. Black-box control for linear dynamical systems. In Conference on Learning Theory , pages 1114--1143. PMLR, 2021

    work page 2021

  27. [27]

    Online linear quadratic control

    Alon Cohen, Avinatan Hasidim, Tomer Koren, Nevena Lazic, Yishay Mansour, and Kunal Talwar. Online linear quadratic control. In International Conference on Machine Learning , pages 1029--1038, 2018

    work page 2018

  28. [28]

    Bandit linear control

    Asaf Cassel and Tomer Koren. Bandit linear control. Advances in Neural Information Processing Systems , 33:8872--8882, 2020

    work page 2020

  29. [29]

    Learning linear-quadratic regulators efficiently with only T regret

    Alon Cohen, Tomer Koren, and Yishay Mansour. Learning linear-quadratic regulators efficiently with only T regret. In International Conference on Machine Learning , pages 1300--1309, 2019

    work page 2019

  30. [30]

    Introduction to algorithms

    Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein. Introduction to algorithms . MIT press, 2022

    work page 2022

  31. [31]

    Bilinear classes: A structural framework for provable generalization in rl

    Simon Du, Sham Kakade, Jason Lee, Shachar Lovett, Gaurav Mahajan, Wen Sun, and Ruosong Wang. Bilinear classes: A structural framework for provable generalization in rl. In International Conference on Machine Learning , pages 2826--2836. PMLR, 2021

    work page 2021

  32. [32]

    Regret bounds for robust adaptive control of the linear quadratic regulator

    Sarah Dean, Horia Mania, Nikolai Matni, Benjamin Recht, and Stephen Tu. Regret bounds for robust adaptive control of the linear quadratic regulator. In Advances in Neural Information Processing Systems , pages 4188--4197, 2018

    work page 2018

  33. [33]

    On the sample complexity of the linear quadratic regulator

    Sarah Dean, Horia Mania, Nikolai Matni, Benjamin Recht, and Stephen Tu. On the sample complexity of the linear quadratic regulator. Foundations of Computational Mathematics , 20(4):633--679, 2020

    work page 2020

  34. [34]

    System level synthesis: A tutorial

    John C Doyle, Nikolai Matni, Yuh-Shyang Wang, James Anderson, and Steven Low. System level synthesis: A tutorial. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC) , pages 2856--2867. IEEE, 2017

    work page 2017

  35. [35]

    Guaranteed margins for lqg regulators

    John C Doyle. Guaranteed margins for lqg regulators. IEEE Transactions on automatic Control , 23(4):756--757, 1978

    work page 1978

  36. [36]

    Experts in a markov decision process

    Eyal Even-Dar, Sham M Kakade, and Yishay Mansour. Experts in a markov decision process. Advances in neural information processing systems , 17, 2004

    work page 2004

  37. [37]

    Control theory: History, mathematical achievements and perspectives

    Enrique Fern \'a ndez Cara and Enrique Zuazua Iriondo. Control theory: History, mathematical achievements and perspectives. Bolet \' n de la Sociedad Espa \ n ola de Matem \'a tica Aplicada, 26, 79-140. , 2003

    work page 2003

  38. [38]

    Global convergence of policy gradient methods for the linear quadratic regulator

    Maryam Fazel, Rong Ge, Sham Kakade, and Mehran Mesbahi. Global convergence of policy gradient methods for the linear quadratic regulator. In International Conference on Machine Learning , pages 1467--1476, 2018

    work page 2018

  39. [39]

    Pac adaptive control of linear systems

    Claude-Nicolas Fiechter. Pac adaptive control of linear systems. In Proceedings of the tenth annual conference on Computational learning theory , pages 72--80, 1997

    work page 1997

  40. [40]

    Foster and Max Simchowitz

    Dylan J. Foster and Max Simchowitz. Logarithmic regret for adversarial online control, 2020

    work page 2020

  41. [41]

    An input--output parametrization of stabilizing controllers: Amidst youla and system level synthesis

    Luca Furieri, Yang Zheng, Antonis Papachristodoulou, and Maryam Kamgarpour. An input--output parametrization of stabilizing controllers: Amidst youla and system level synthesis. IEEE Control Systems Letters , 3(4):1014--1019, 2019

    work page 2019

  42. [42]

    Best of both worlds in online control: Competitive ratio and policy regret

    Gautam Goel, Naman Agarwal, Karan Singh, and Elad Hazan. Best of both worlds in online control: Competitive ratio and policy regret. arXiv preprint arXiv:2211.11219 , 2022

    work page arXiv 2022

  43. [43]

    Competitive control

    Gautam Goel and Babak Hassibi. Competitive control. arXiv preprint arXiv:2107.13657 , 2021

    work page arXiv 2021

  44. [44]

    Non-stochastic control with bandit feedback

    Paula Gradu, John Hallman, and Elad Hazan. Non-stochastic control with bandit feedback. Advances in Neural Information Processing Systems , 33:10764--10774, 2020

    work page 2020

  45. [45]

    Adaptive regret for control of time-varying dynamics

    Paula Gradu, Elad Hazan, and Edgar Minasyan. Adaptive regret for control of time-varying dynamics. arXiv preprint arXiv:2007.04393 , 2020

    work page arXiv 2007

  46. [46]

    Beyond online balanced descent: An optimal algorithm for smoothed online optimization

    Gautam Goel, Yiheng Lin, Haoyuan Sun, and Adam Wierman. Beyond online balanced descent: An optimal algorithm for smoothed online optimization. Advances in Neural Information Processing Systems , 32, 2019

    work page 2019

  47. [47]

    An online algorithm for smoothed regression and lqr control

    Gautam Goel and Adam Wierman. An online algorithm for smoothed regression and lqr control. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 2504--2513. PMLR, 2019

    work page 2019

  48. [48]

    Introduction to online convex optimization

    Elad Hazan. Introduction to online convex optimization. Foundations and Trends in Optimization , 2(3-4):157--325, 2016

    work page 2016

  49. [49]

    Lecture notes: Optimization for machine learning, 2019

    Elad Hazan. Lecture notes: Optimization for machine learning, 2019

    work page 2019

  50. [50]

    Linear systems theory

    Joao P Hespanha. Linear systems theory . Princeton university press, 2018

    work page 2018

  51. [51]

    Effective construction of linear state-variable models from input/output functions

    BL HO and Rudolf E K \'a lm \'a n. Effective construction of linear state-variable models from input/output functions. at-Automatisierungstechnik , 14(1-12):545--548, 1966

    work page 1966

  52. [52]

    The nonstochastic control problem

    Elad Hazan, Sham Kakade, and Karan Singh. The nonstochastic control problem. In Algorithmic Learning Theory , pages 408--421. PMLR, 2020

    work page 2020

  53. [53]

    Spectral filtering for general linear dynamical systems

    Elad Hazan, Holden Lee, Karan Singh, Cyril Zhang, and Yi Zhang. Spectral filtering for general linear dynamical systems. In Advances in Neural Information Processing Systems , pages 4634--4643, 2018

    work page 2018

  54. [54]

    Gradient Descent Learns Linear Dynamical Systems

    Moritz Hardt, Tengyu Ma, and Benjamin Recht. Gradient descent learns linear dynamical systems. arXiv preprint arXiv:1609.05191 , 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  55. [55]

    Icml tutorial: Online and nonstochastic control

    Elad Hazan and Karan Singh. Icml tutorial: Online and nonstochastic control. In International Conference on Machine Learning , 2021

    work page 2021

  56. [56]

    Linear estimation in krein spaces

    Babak Hassibi, Ali H Sayed, and Thomas Kailath. Linear estimation in krein spaces. ii. applications. IEEE Transactions on Automatic Control , 41(1):34--49, 1996

    work page 1996

  57. [57]

    Learning linear dynamical systems via spectral filtering

    Elad Hazan, Karan Singh, and Cyril Zhang. Learning linear dynamical systems via spectral filtering. In Advances in Neural Information Processing Systems , pages 6702--6712, 2017

    work page 2017

  58. [58]

    Efficient reinforcement learning for high dimensional linear quadratic systems

    Morteza Ibrahimi, Adel Javanmard, and Benjamin Roy. Efficient reinforcement learning for high dimensional linear quadratic systems. Advances in Neural Information Processing Systems , 25, 2012

    work page 2012

  59. [59]

    Contextual decision processes with low bellman rank are pac-learnable

    Nan Jiang, Akshay Krishnamurthy, Alekh Agarwal, John Langford, and Robert E Schapire. Contextual decision processes with low bellman rank are pac-learnable. In International Conference on Machine Learning , pages 1704--1713. PMLR, 2017

    work page 2017

  60. [60]

    Provably efficient reinforcement learning with linear function approximation

    Chi Jin, Zhuoran Yang, Zhaoran Wang, and Michael I Jordan. Provably efficient reinforcement learning with linear function approximation. In Conference on Learning Theory , pages 2137--2143. PMLR, 2020

    work page 2020

  61. [61]

    A new approach to linear filtering and prediction problems

    Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. Journal of Basic Engineering , 82.1:35--45, 1960

    work page 1960

  62. [62]

    Nonlinear control , volume 406

    Hassan K Khalil. Nonlinear control , volume 406. Pearson New York, 2015

    work page 2015

  63. [63]

    Information theoretic regret bounds for online nonlinear control

    Sham Kakade, Akshay Krishnamurthy, Kendall Lowrey, Motoya Ohnishi, and Wen Sun. Information theoretic regret bounds for online nonlinear control. Advances in Neural Information Processing Systems , 33:15312--15325, 2020

    work page 2020

  64. [64]

    On-line learning of linear dynamical systems: Exponential forgetting in kalman filters

    Mark Kozdoba, Jakub Marecek, Tigran Tchrakian, and Shie Mannor. On-line learning of linear dynamical systems: Exponential forgetting in kalman filters. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 4098--4105, 2019

    work page 2019

  65. [65]

    Logarithmic regret bound in partially observable linear dynamical systems, 2020

    Sahin Lale, Kamyar Azizzadenesheli, Babak Hassibi, and Anima Anandkumar. Logarithmic regret bound in partially observable linear dynamical systems, 2020

    work page 2020

  66. [66]

    Regret bound of adaptive control in linear quadratic gaussian (lqg) systems, 2020

    Sahin Lale, Kamyar Azizzadenesheli, Babak Hassibi, and Anima Anandkumar. Regret bound of adaptive control in linear quadratic gaussian (lqg) systems, 2020

    work page 2020

  67. [67]

    Regret minimization in partially observable linear quadratic control, 2020

    Sahin Lale, Kamyar Azizzadenesheli, Babak Hassibi, and Anima Anandkumar. Regret minimization in partially observable linear quadratic control, 2020

    work page 2020

  68. [68]

    Online optimal control with affine constraints

    Yingying Li, Subhro Das, and Na Li. Online optimal control with affine constraints. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 8527--8537, 2021

    work page 2021

  69. [69]

    A generalized iterative lqg method for locally-optimal feedback control of constrained nonlinear stochastic systems

    Weiwei Li. A generalized iterative lqg method for locally-optimal feedback control of constrained nonlinear stochastic systems. In Proceedings of the 2005, American Control Conference, 2005. , pages 300--306. IEEE, 2005

    work page 2005

  70. [70]

    Guided policy search

    Sergey Levine and Vladlen Koltun. Guided policy search. In International conference on machine learning , pages 1--9. PMLR, 2013

    work page 2013

  71. [71]

    Edward N. Lorenz. D eterministic N onperiodic F low. Journal of the Atmospheric Sciences , 20(2):130--141, mar 1963

    work page 1963

  72. [72]

    The general problem of the stability of motion

    Aleksandr Mikhailovich Lyapunov. The general problem of the stability of motion. International journal of control , 55(3):531--534, 1992

    work page 1992

  73. [73]

    James Clerk Maxwell. I. on governors. Proceedings of the Royal Society of London , 16:270--283, 1868

    work page

  74. [74]

    Online control of unknown time-varying dynamical systems

    Edgar Minasyan, Paula Gradu, Max Simchowitz, and Elad Hazan. Online control of unknown time-varying dynamical systems. Advances in Neural Information Processing Systems , 34:15934--15945, 2021

    work page 2021

  75. [75]

    Certainty equivalence is efficient for linear quadratic control

    Horia Mania, Stephen Tu, and Benjamin Recht. Certainty equivalence is efficient for linear quadratic control. In Advances in Neural Information Processing Systems , pages 10154--10164, 2019

    work page 2019

  76. [76]

    Non-asymptotic identification of lti systems from a single trajectory

    Samet Oymak and Necmiye Ozay. Non-asymptotic identification of lti systems from a single trajectory. In 2019 American control conference (ACC) , pages 5655--5661. IEEE, 2019

    work page 2019

  77. [77]

    Geometric exploration for online control

    Orestis Plevrakis and Elad Hazan. Geometric exploration for online control. Advances in Neural Information Processing Systems , 33:7637--7647, 2020

    work page 2020

  78. [78]

    Artificial intelligence: A modern approach

    P R Russel and S A Norvig. Artificial intelligence: A modern approach. Prentice Hall Upper Saddle River, NJ, USA: Rani, M., Nayak, R., & Vyas, OP (2015). An ontology-based adaptive personalized e-learning system, assisted by software agents on cloud storage. Knowledge-Based Systems , 90:33--48, 2002

    work page 2015

  79. [79]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction . The MIT Press, second edition, 2018

    work page 2018

  80. [80]

    Adaptive control: stability, convergence, and robustness, 1990

    Shankar Sastry, Marc Bodson, and James F Bartram. Adaptive control: stability, convergence, and robustness, 1990

    work page 1990

Showing first 80 references.