Pith. sign in

REVIEW 1 major objections 2 minor 52 references

AdaDQN decouples the allowable stepsize from the number of local updates in decentralized nonconvex optimization.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-27 15:48 UTC pith:VNAQ337V

load-bearing objection AdaDQN claims a decoupled stepsize bound via its RIA framework and adaptive termination, but the decoupling may still depend on whether the local error analysis avoids reintroducing 1/K factors. the 1 major comments →

arxiv 2606.09070 v1 pith:VNAQ337V submitted 2026-06-08 math.OC

Beyond Fixed Local Updates: An Adaptive Decentralized Quasi-Newton Method Free of Stepsize Degradation

Hao Wu , Liping Wang This is my paper
classification math.OC
keywords decentralized optimizationquasi-Newton methodnonconvex optimizationlocal updatesmajorization-minimizationconvergence analysisadaptive algorithmsevent-triggered communication
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces an adaptive decentralized quasi-Newton method called AdaDQN that solves smooth nonconvex problems over undirected networks. It revisits local-update schemes through a majorization-minimization lens to build a Robust Inexact Algorithm framework, then uses a consensus-aware termination rule and memoryless BFGS update to keep convergence guarantees intact. The key result is that the stepsize bound no longer has to shrink inversely with the maximum number of local updates per round. This removes a long-standing theoretical limit on how much local computation can be performed before communicating.

Core claim

AdaDQN achieves global convergence to a first-order stationary point for smooth nonconvex decentralized problems, with the convergence stepsize bound proven to be independent of the reciprocal of the maximum number of local updates, through integration of a safeguarded termination criterion, scalable BFGS update, and event-triggered communication within the RIA framework.

What carries the argument

The Robust Inexact Algorithm (RIA) framework derived from the Majorization-Minimization lens, which analyzes the effect of inexact local updates under the chosen termination criterion and BFGS rule.

Load-bearing premise

The Robust Inexact Algorithm framework must hold for the specific local-update termination criterion and BFGS update used in AdaDQN.

What would settle it

An explicit counterexample or numerical instance in which increasing the maximum number of local updates forces a strictly smaller stepsize to maintain convergence on a smooth nonconvex problem would falsify the decoupling claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Global convergence to first-order stationary points holds without the previous stepsize penalty from extra local work.
  • Communication rounds can be reduced via the event-triggered protocol while preserving the same theoretical stepsize range.
  • The method applies directly to undirected networks and yields a better computation-communication tradeoff than prior decentralized schemes.
  • The RIA framework supplies a template for designing other local-update methods whose stepsize bounds remain stable.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The decoupling result may extend to directed networks or stochastic gradients if the RIA analysis can be adapted.
  • In large-scale settings the independence from local-update count could allow agents to run many cheap iterations before any communication without retuning stepsizes.
  • Similar majorization-minimization arguments might remove analogous bottlenecks in other inexact or asynchronous optimization algorithms.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 2 minor

Summary. The paper proposes AdaDQN, an adaptive decentralized quasi-Newton method for smooth nonconvex optimization over undirected networks. It revisits local-update methods via a Majorization-Minimization lens to derive a Robust Inexact Algorithm (RIA) framework, then uses a safeguarded consensus-aware termination criterion, memoryless BFGS updates, and event-triggered communication. The central claim is global convergence to a first-order stationary point with a stepsize bound that is decoupled from the reciprocal of the maximum number of local updates (overcoming the standard 1/K bottleneck). Experiments show improved computation-communication tradeoffs versus prior decentralized methods.

Significance. If the decoupling result holds, the work would be significant for decentralized optimization: it removes a pessimistic theoretical restriction that has limited the practical use of multiple local updates. The RIA framework derived from the MM lens, together with the explicit construction of a termination criterion that preserves the required inexactness without reintroducing K-dependent factors, would constitute a technical contribution. The memoryless BFGS and event-triggered protocol are pragmatic additions that could translate to measurable gains in communication cost.

major comments (1)
  1. [§4] §4 (convergence analysis) and the RIA framework definition: the proof that the stepsize upper bound is independent of 1/K rests on the local-update error and consensus-error terms satisfying the RIA inexactness conditions without producing a multiplicative K factor in the descent inequality or effective Lipschitz constant. The manuscript must explicitly verify (via the specific termination criterion and memoryless BFGS update) that no such factor reappears; otherwise the claimed decoupling does not follow.
minor comments (2)
  1. [Algorithm 1] Notation for the termination threshold and the event-trigger parameter should be introduced once and used consistently; currently the same symbol appears to be overloaded in the algorithm box and the analysis.
  2. [§5] The experimental section would benefit from reporting the actual number of local updates per iteration (not just the maximum allowed) so readers can directly assess the computation-communication tradeoff achieved.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback and for acknowledging the potential significance of the decoupling result. We address the single major comment below.

read point-by-point responses

  1. Referee: [§4] §4 (convergence analysis) and the RIA framework definition: the proof that the stepsize upper bound is independent of 1/K rests on the local-update error and consensus-error terms satisfying the RIA inexactness conditions without producing a multiplicative K factor in the descent inequality or effective Lipschitz constant. The manuscript must explicitly verify (via the specific termination criterion and memoryless BFGS update) that no such factor reappears; otherwise the claimed decoupling does not follow.

    Authors: We agree that explicit verification is essential for the claimed decoupling. In the proof of the main convergence result (Theorem 4.1), the safeguarded consensus-aware termination criterion is constructed so that the consensus error is bounded by a quantity independent of the maximum number of local updates K; this bound is then substituted directly into the RIA inexactness condition without introducing a multiplicative K. The memoryless BFGS update is shown to satisfy the required curvature and descent conditions while preserving the same inexactness tolerance, again without K-dependent accumulation. To make this verification more prominent, we will add a dedicated remark immediately after the proof that isolates the relevant error bounds and confirms the absence of any K factor in the descent inequality or effective Lipschitz constant. revision: partial

Circularity Check

0 steps flagged

No circularity: RIA framework and stepsize decoupling derived independently from MM lens

full rationale

The paper derives the RIA framework directly from the Majorization-Minimization lens within this work, then applies it to prove global convergence and the decoupling of the convergence stepsize bound from 1/max-local-updates for the specific termination criterion and BFGS update. No equations or claims reduce a result to a fitted parameter, self-citation, or input by construction; the central claim rests on the new framework rather than prior fitted constants or renamed patterns. The derivation is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 1 invented entities

The central claim rests on standard domain assumptions for decentralized nonconvex optimization plus the newly introduced RIA framework; no free parameters or invented physical entities are mentioned.

axioms (2)
  • domain assumption Objective functions are smooth
    Required for quasi-Newton updates and global convergence to first-order stationary points in nonconvex setting.
  • domain assumption Communication network is undirected and connected
    Standard assumption for consensus-based decentralized algorithms over networks.
invented entities (1)
  • Robust Inexact Algorithm (RIA) framework no independent evidence
    purpose: Theoretical lens to analyze local-update methods and derive the adaptive termination criterion
    Introduced in the paper to overcome the stepsize degradation; no independent evidence outside this work.

pith-pipeline@v0.9.1-grok · 5709 in / 1399 out tokens · 19551 ms · 2026-06-27T15:48:59.643067+00:00 · methodology

0 comments
read the original abstract

This paper proposes a novel Adaptive Decentralized Quasi-Newton (AdaDQN) method for solving smooth nonconvex optimization problems over undirected networks. While existing decentralized algorithms with multiple local updates suffer from a pessimistic theoretical bottleneck, where the convergence stepsize must be inversely proportional to the number of local updates, our work overcomes this limitation. We revisit local update methods through a Majorization-Minimization (MM) lens and establish a Robust Inexact Algorithm (RIA) framework. Inspired by this framework, AdaDQN integrates a safeguarded consensus-aware termination criterion to dynamically balance local computational gains against consensus error, complemented by a scalable memoryless BFGS update and an event-triggered communication protocol to significantly reduce communication overhead. We establish the global convergence of AdaDQN to a first-order stationary point. Crucially, we prove that the convergence stepsize bound is decoupled from the reciprocal of maximum number of local updates. Numerical experiments demonstrate that AdaDQN achieves a superior computation-communication tradeoff, outperforming state-of-the-art decentralized methods across various performance metrics.

Figures

Figures reproduced from arXiv: 2606.09070 by Hao Wu, Liping Wang.

Figure 5.1
Figure 5.1. Figure 5.1: Optimality error of comparison algorithms for minimizing the nonconvex [PITH_FULL_IMAGE:figures/full_fig_p029_5_1.png] view at source ↗
Figure 5.2
Figure 5.2. Figure 5.2: Optimality error of comparison algorithms for minimizing the nonconvex [PITH_FULL_IMAGE:figures/full_fig_p029_5_2.png] view at source ↗
Figure 5.3
Figure 5.3. Figure 5.3: Optimality error of comparison algorithms for minimizing the nonconvex [PITH_FULL_IMAGE:figures/full_fig_p030_5_3.png] view at source ↗
Figure 5.4
Figure 5.4. Figure 5.4: Optimality error of comparison algorithms for minimizing the nonconvex [PITH_FULL_IMAGE:figures/full_fig_p030_5_4.png] view at source ↗
Figure 5.5
Figure 5.5. Figure 5.5: Balance between gradient evaluation number and communication volume [PITH_FULL_IMAGE:figures/full_fig_p030_5_5.png] view at source ↗
Figure 5.6
Figure 5.6. Figure 5.6: Optimality error of comparison algorithms for minimizing the nonconvex [PITH_FULL_IMAGE:figures/full_fig_p031_5_6.png] view at source ↗
Figure 5.7
Figure 5.7. Figure 5.7: Optimality error of comparison algorithms for minimizing the nonconvex [PITH_FULL_IMAGE:figures/full_fig_p031_5_7.png] view at source ↗
Figure 5.8
Figure 5.8. Figure 5.8: Optimality error of comparison algorithms for minimizing the nonconvex [PITH_FULL_IMAGE:figures/full_fig_p031_5_8.png] view at source ↗
Figure 5.9
Figure 5.9. Figure 5.9: Optimality error of comparison algorithms for minimizing the nonconvex [PITH_FULL_IMAGE:figures/full_fig_p032_5_9.png] view at source ↗
Figure 5.10
Figure 5.10. Figure 5.10: Optimality error of comparison algorithms for minimizing the nonconvex [PITH_FULL_IMAGE:figures/full_fig_p032_5_10.png] view at source ↗
Figure 5.11
Figure 5.11. Figure 5.11: Optimality error of comparison algorithms for minimizing the nonconvex [PITH_FULL_IMAGE:figures/full_fig_p032_5_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

52 extracted references · 4 canonical work pages · 2 internal anchors

  1. [1]

    A decentralized approach for voltage control by multiple distributed energy resources,

    G. Fusco and M. Russo, “A decentralized approach for voltage control by multiple distributed energy resources,”IEEE Transactions on Smart Grid, vol. 12, no. 4, pp. 3115–3127, 2021

    2021

  2. [2]

    Asynchronous decentralized learning over unreliable wireless networks,

    E. Jeong, M. Zecchin, and M. Kountouris, “Asynchronous decentralized learning over unreliable wireless networks,” inInternational Conference on Communications, 2022, pp. 607–612

    2022

  3. [3]

    Distributed reptile algorithm for meta-learning over multi-agent systems,

    X. Zhang, C. Hu, B. He, and Z. Han, “Distributed reptile algorithm for meta-learning over multi-agent systems,”IEEE Transactions on Signal Processing, vol. 70, pp. 5443–5456, 2022

    2022

  4. [4]

    Fully distributed robust reserve scheduling for coupled transmission and distribution systems,

    Z. Chen, Z. Li, C. Guo, J. Wang, and Y. Ding, “Fully distributed robust reserve scheduling for coupled transmission and distribution systems,”IEEE Transactions on Power Systems, vol. 36, no. 1, pp. 169–182, 2020

    2020

  5. [5]

    Robust aggregation for federated learning,

    K. Pillutla, S. M. Kakade, and Z. Harchaoui, “Robust aggregation for federated learning,” IEEE Transactions on Signal Processing, vol. 70, pp. 1142–1154, 2022

    2022

  6. [6]

    Distributed subgradient methods for multi-agent optimization,

    A. Nedi´ c and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” IEEE Transactions on Automatic Control, vol. 54, no. 1, pp. 48–61, 2009

    2009

  7. [7]

    On the convergence of decentralized gradient descent,

    K. Yuan, Q. Ling, and W. Yin, “On the convergence of decentralized gradient descent,”SIAM Journal on Optimization, vol. 26, no. 3, pp. 1835–1854, 2016

    2016

  8. [8]

    On nonconvex decentralized gradient descent,

    J. Zeng and W. Yin, “On nonconvex decentralized gradient descent,”IEEE Transactions on Signal Processing, vol. 66, no. 11, pp. 2834–2848, 2018

    2018

  9. [9]

    Augmented distributed gradient methods for multi- agent optimization under uncoordinated constant stepsizes,

    J. Xu, S. Zhu, Y. C. Soh, and L. Xie, “Augmented distributed gradient methods for multi- agent optimization under uncoordinated constant stepsizes,” inConference on Decision and Control, 2015, pp. 2055–2060

    2015

  10. [10]

    Harnessing smoothness to accelerate distributed optimization,

    G. Qu and N. Li, “Harnessing smoothness to accelerate distributed optimization,”IEEE Trans- actions on Control of Network Systems, vol. 5, no. 3, pp. 1245–1260, 2017

    2017

  11. [11]

    Achieving geometric convergence for distributed opti- mization over time-varying graphs,

    A. Nedi´ c, A. Olshevsky, and W. Shi, “Achieving geometric convergence for distributed opti- mization over time-varying graphs,”SIAM Journal on Optimization, vol. 27, no. 4, pp. 2597–2633, 2017

    2017

  12. [12]

    Geometrically convergent distributed opti- mization with uncoordinated step-sizes,

    A. Nedi´ c, A. Olshevsky, W. Shi, and C. A. Uribe, “Geometrically convergent distributed opti- mization with uncoordinated step-sizes,” inAmerican Control Conference. IEEE, 2017, pp. 3950–3955

    2017

  13. [13]

    Distributed heavy-ball: A generalization and acceleration of first- order methods with gradient tracking,

    R. Xin and U. A. Khan, “Distributed heavy-ball: A generalization and acceleration of first- order methods with gradient tracking,”IEEE Transactions on Automatic Control, vol. 65, no. 6, pp. 2627–2633, 2019

    2019

  14. [14]

    Achieving geometric convergence for distributed optimization with barzilai-borwein step sizes,

    J. Gao, X.-W. Liu, Y.-H. Dai, Y. Huang, and P. Yang, “Achieving geometric convergence for distributed optimization with barzilai-borwein step sizes,”Science China. Information Sciences, vol. 65, no. 4, p. 149204, 2022

    2022

  15. [15]

    Distributed dual gradient tracking for resource allocation in unbalanced networks,

    J. Zhang, K. You, and K. Cai, “Distributed dual gradient tracking for resource allocation in unbalanced networks,”IEEE Transactions on Signal Processing, vol. 68, pp. 2186–2198, 2020

    2020

  16. [16]

    Optimal gradient tracking for decentralized optimization,

    Z. Song, L. Shi, S. Pu, and M. Yan, “Optimal gradient tracking for decentralized optimization,” Mathematical Programming, vol. 207, no. 1, pp. 1–53, 2024

    2024

  17. [17]

    Extra: An exact first-order algorithm for decentralized consensus optimization,

    W. Shi, Q. Ling, G. Wu, and W. Yin, “Extra: An exact first-order algorithm for decentralized consensus optimization,”SIAM Journal on Optimization, vol. 25, no. 2, pp. 944–966, 2015

    2015

  18. [18]

    Exact diffusion for distributed optimization and learning—part i: Algorithm development,

    K. Yuan, B. Ying, X. Zhao, and A. H. Sayed, “Exact diffusion for distributed optimization and learning—part i: Algorithm development,”IEEE Transactions on Signal Processing, vol. 67, no. 3, pp. 708–723, 2018

    2018

  19. [19]

    Local exact-diffusion for decentralized optimization and learning,

    S. A. Alghunaim, “Local exact-diffusion for decentralized optimization and learning,”IEEE Transactions on Automatic Control, vol. 69, no. 11, pp. 7371–7386, 2024

    2024

  20. [20]

    Music: Accelerated convergence for distributed opti- mization with inexact and exact methods,

    M. Wu, H. Liao, Z. Ding, and Y. Xiao, “Music: Accelerated convergence for distributed opti- mization with inexact and exact methods,”IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 3, pp. 4893–4907, 2025. 34

    2025

  21. [21]

    On the linear convergence of the admm in decentralized consensus optimization,

    W. Shi, Q. Ling, K. Yuan, G. Wu, and W. Yin, “On the linear convergence of the admm in decentralized consensus optimization,”IEEE Transactions on Signal Processing, vol. 62, no. 7, pp. 1750–1761, 2014

    2014

  22. [22]

    Primal–dual methods for large-scale and distributed convex optimization and data analytics,

    D. Jakoveti´ c, D. Bajovi´ c, J. Xavier, and J. M. Moura, “Primal–dual methods for large-scale and distributed convex optimization and data analytics,”Proceedings of the IEEE, vol. 108, no. 11, pp. 1923–1938, 2020

    1923

  23. [23]

    A decentralized primal-dual framework for non-convex smooth consensus optimization,

    G. Mancino-Ball, Y. Xu, and J. Chen, “A decentralized primal-dual framework for non-convex smooth consensus optimization,”IEEE Transactions on Signal Processing, vol. 71, pp. 525–538, 2023

    2023

  24. [24]

    Balancing communication and com- putation in distributed optimization,

    A. S. Berahas, R. Bollapragada, N. S. Keskar, and E. Wei, “Balancing communication and com- putation in distributed optimization,”IEEE Transactions on Automatic Control, vol. 64, no. 8, pp. 3141–3155, 2018

    2018

  25. [25]

    Gradient tracking with multiple local sgd for decentralized non-convex learning,

    S. Ge and T.-H. Chang, “Gradient tracking with multiple local sgd for decentralized non-convex learning,” inConference on Decision and Control. IEEE, 2023, pp. 133–138

    2023

  26. [26]

    On the performance of gradient tracking with local updates,

    E. D. H. Nguyen, S. A. Alghunaim, K. Yuan, and C. A. Uribe, “On the performance of gradient tracking with local updates,” inConference on Decision and Control. IEEE, 2023, pp. 4309–4313

    2023

  27. [27]

    Decentralized gradient tracking with local steps,

    Y. Liu, T. Lin, A. Koloskova, and S. U. Stich, “Decentralized gradient tracking with local steps,”Optimization Methods and Software, pp. 1–28, 2024

    2024

  28. [28]

    Guaranteeing both consensus and optimality in decentralized nonconvex optimization with multiple local updates,

    J. Liu, Z. Wang, and Y. Wang, “Guaranteeing both consensus and optimality in decentralized nonconvex optimization with multiple local updates,” 2025. [Online]. Available: https://arxiv.org/abs/2511.05242

    work page arXiv 2025

  29. [29]

    Distributed stochastic gradient descent with event-triggered com- munication,

    J. George and P. Gurram, “Distributed stochastic gradient descent with event-triggered com- munication,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, 2020, pp. 7169–7178

    2020

  30. [30]

    An event-triggered approach for gradient tracking in consensus-based distributed optimization,

    L. Gao, S. Deng, H. Li, and C. Li, “An event-triggered approach for gradient tracking in consensus-based distributed optimization,”IEEE Transactions on Network Science and Engineering, vol. 9, no. 2, pp. 510–523, 2021

    2021

  31. [31]

    Event-triggered distributed nonconvex optimization with progress-based threshold,

    C. Liu and E. Shi, “Event-triggered distributed nonconvex optimization with progress-based threshold,” inConference on Decision and Control. IEEE, 2023, pp. 309–314

    2023

  32. [32]

    Distributed event-triggered nonconvex optimization under polyak- lojasiewicz condition,

    C. Gao, L. Xu, K. Zhang, Y. Li, Z. Liu, and T. Yang, “Distributed event-triggered nonconvex optimization under polyak- lojasiewicz condition,” inInternational Conference on Control, Automation, Robotics and Vision. IEEE, 2024, pp. 930–935

    2024

  33. [33]

    Distributed online adaptive gradient descent with event-triggered communication,

    K. Okamoto, N. Hayashi, and S. Takai, “Distributed online adaptive gradient descent with event-triggered communication,”IEEE Transactions on Control of Network Systems, vol. 11, no. 2, pp. 610–622, 2023

    2023

  34. [34]

    Distributed nonconvex opti- mization with event-triggered communication,

    L. Xu, X. Yi, Y. Shi, K. H. Johansson, T. Chai, and T. Yang, “Distributed nonconvex opti- mization with event-triggered communication,”IEEE Transactions on Automatic Control, vol. 69, no. 4, pp. 2745–2752, 2023

    2023

  35. [35]

    Communication-censored admm for decentralized consensus optimization,

    Y. Liu, W. Xu, G. Wu, Z. Tian, and Q. Ling, “Communication-censored admm for decentralized consensus optimization,”IEEE Transactions on Signal Processing, vol. 67, no. 10, pp. 2565–2579, 2019

    2019

  36. [36]

    Communication-censored linearized admm for decentral- ized consensus optimization,

    W. Li, Y. Liu, Z. Tian, and Q. Ling, “Communication-censored linearized admm for decentral- ized consensus optimization,”IEEE Transactions on Signal and Information Processing over Networks, vol. 6, pp. 18–34, 2019

    2019

  37. [37]

    Dqc-admm: Decentralized dynamic admm with quantized and censored communications,

    Y. Liu, G. Wu, Z. Tian, and Q. Ling, “Dqc-admm: Decentralized dynamic admm with quantized and censored communications,”IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 8, pp. 3290–3304, 2021

    2021

  38. [38]

    Decentralized admm with compressed and event-triggered communication,

    Z. Zhang, S. Yang, and W. Xu, “Decentralized admm with compressed and event-triggered communication,”Neural Networks, 2023

    2023

  39. [39]

    A general framework of exact primal-dual first-order algorithms for distributed optimization,

    F. Mansoori and E. Wei, “A general framework of exact primal-dual first-order algorithms for distributed optimization,” inConference on Decision and Control. IEEE, 2019, pp. 6386–6391

    2019

  40. [40]

    On the convergence of nested decentralized gradient methods with multiple consensus and gradient steps,

    A. S. Berahas, R. Bollapragada, and E. Wei, “On the convergence of nested decentralized gradient methods with multiple consensus and gradient steps,”IEEE Transactions on Signal Processing, vol. 69, pp. 4192–4203, 2021

    2021

  41. [41]

    Decentralized federated learning: Balancing communication and computing costs,

    W. Liu, L. Chen, and W. Zhang, “Decentralized federated learning: Balancing communication and computing costs,”IEEE Transactions on Signal and Information Processing over Networks, vol. 8, pp. 131–143, 2022

    2022

  42. [42]

    On the computation-communication trade-off with a flexible gradient tracking approach,

    Y. Huang and J. Xu, “On the computation-communication trade-off with a flexible gradient tracking approach,” inConference on Decision and Control. IEEE, 2023, pp. 284–289

    2023

  43. [43]

    Balancing communication and computation 35 in gradient tracking algorithms for decentralized optimization,

    A. S. Berahas, R. Bollapragada, and S. Gupta, “Balancing communication and computation 35 in gradient tracking algorithms for decentralized optimization,”Journal of Optimization Theory and Applications, pp. 1–34, 2024

    2024

  44. [44]

    Pareto-optimal trade-offs between communication and computation with flexible gradient tracking,

    Y. Huang, J. Xu, L. Chai, J. Chen, and K. H. Johansson, “Pareto-optimal trade-offs between communication and computation with flexible gradient tracking,” 2026. [Online]. Available: https://arxiv.org/abs/2509.18129

    work page arXiv 2026

  45. [45]

    MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems

    T. Chen, M. Li, Y. Li, M. Lin, N. Wang, M. Wang, T. Xiao, B. Xu, C. Zhang, and Z. Zhang, “Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems,” 2015. [Online]. Available: https://arxiv.org/abs/1512.01274

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  46. [46]

    Gnsd: A gradient-tracking based nonconvex stochastic algorithm for decentralized optimization,

    S. Lu, X. Zhang, H. Sun, and M. Hong, “Gnsd: A gradient-tracking based nonconvex stochastic algorithm for decentralized optimization,” inData Science Workshop. IEEE, 2019, pp. 315–321

    2019

  47. [47]

    Diffusion adaptation over networks,

    A. H. Sayed, “Diffusion adaptation over networks,” inAcademic Press Library in Signal Pro- cessing, 2014, vol. 3, pp. 323–453

    2014

  48. [48]

    Distributed average consensus with least-mean-square devi- ation,

    L. Xiao, S. Boyd, and S.-J. Kim, “Distributed average consensus with least-mean-square devi- ation,”Journal of Parallel and Distributed Computing, vol. 67, no. 1, pp. 33–46, 2007

    2007

  49. [49]

    Discrete-time dynamic average consensus,

    M. Zhu and S. Mart´ ınez, “Discrete-time dynamic average consensus,”Automatica, vol. 46, no. 2, pp. 322–329, 2010

    2010

  50. [50]

    A Unified Algorithm for Nonconvex Decentralized Nonlinear Optimization

    H. Wu, L. Wang, and H. Zhang, “A unified algorithm for nonconvex decentralized nonlinear optimization,” 2026. [Online]. Available: https://arxiv.org/abs/2511.19182

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  51. [51]

    Scaffold: Stochas- tic controlled averaging for federated learning,

    S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “Scaffold: Stochas- tic controlled averaging for federated learning,” inInternational conference on machine learning. PMLR, 2020, pp. 5132–5143

    2020

  52. [52]

    On the convergence of fedavg on non-iid data,

    X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-iid data,” inInternational Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=HJxNAnVtDS Appendix A. Analytical tools. Two analytical tools are provided below. Lemma A.1.(Young’s inequality) For any two vectorsv 1,v 2 ∈R p,...

    2020