Pith. sign in

REVIEW 1 major objections 1 minor 24 references

Convex compositional energy models trained on small subproblems transfer to larger combinatorial instances without retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 16:13 UTC pith:RYUOZNNE

load-bearing objection The paper pairs input-convex networks with a convex relaxation to make compositional energy models optimizable by deterministic first-order methods and claims transfer from small training instances to larger ones. the 1 major comments →

arxiv 2605.23395 v2 pith:RYUOZNNE submitted 2026-05-22 cs.LG

Convex Compositional Reasoning Models

Meir Roketlishvili , Semyon Semenov , Maksim Bobrin , Viktor Kovalchuk , Albert Baichorov , Abduragim Shtanchaev , Fakhri Karray , Dmitry V. Dylov
show 2 more authors
Martin Tak\'a\v{c} Arip Asadulaev
This is my paper
classification cs.LG
keywords compositional energy-based modelsconvex optimizationinput-convex neural networkscombinatorial reasoningenergy minimizationtransfer across problem sizes
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that compositional energy-based models fail to scale mainly because of non-convex energy landscapes rather than composition itself. It introduces Convex Compositional Energy Minimization, which uses input-convex networks for each factor and optimizes the sum over a tight convex relaxation of the feasible set. Convexity is preserved under addition, so the relaxed objective supports deterministic first-order methods. Training proceeds in two stages: contrastive learning on individual factors, then end-to-end refinement via an unrolled projected solver. Experiments indicate that models learned on small cases or single sizes apply directly to larger problems.

Core claim

Compositional energy-based models can generalize to larger combinatorial reasoning problems by reusing a learned factor energy across many local constraints, yet the key bottleneck is the non-convex geometry of the learned energy landscape. CCEM parameterizes each factor with an input-convex neural network and optimizes the composed energy over a tight convex relaxation of the feasible set. Because convexity is preserved under summation, the global relaxed objective remains convex, enabling deterministic projected first-order optimization. The framework is trained in two stages—factor-level contrastive learning followed by end-to-end refinement through an unrolled projected solver—and models

What carries the argument

Input-convex neural network factors whose energies are summed and optimized over a tight convex relaxation of the feasible set, preserving convexity for deterministic projected first-order methods.

Load-bearing premise

The solutions obtained from the convex relaxed problem remain useful when the original non-convex constraints are restored at larger scales.

What would settle it

Train a model on small instances of a combinatorial problem such as graph matching or scheduling, then measure whether its objective values and feasibility rates on instances several times larger match or exceed those of models trained directly on the large instances.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Models trained once on small subproblems apply to larger instances without retraining.
  • The composed objective supports reliable deterministic optimization rather than stochastic search.
  • Local factor energies can be shaped independently before global composition refinement.
  • The same architecture works across different problem sizes once the convex relaxation is fixed.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same convexity-preserving construction might apply to other structured prediction tasks where factor graphs are used.
  • If the relaxation tightness holds at scale, end-to-end training on large instances could become unnecessary for many combinatorial domains.
  • The two-stage procedure separates the problem of shaping individual constraints from the problem of composing them globally.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. The paper introduces Convex Compositional Energy Minimization (CCEM) for compositional energy-based models. Each factor is parameterized by an input-convex neural network, and the composed energy is optimized over a tight convex relaxation of the feasible set. Convexity is preserved under summation, enabling deterministic projected first-order optimization. Training occurs in two stages (factor-level contrastive learning followed by end-to-end refinement via an unrolled solver). Experiments claim that models trained on small subproblems or a single problem size transfer to larger instances without retraining.

Significance. If the transfer result holds, the framework could enable reliable scaling of compositional reasoning to larger combinatorial problems by converting non-convex landscapes into convex ones amenable to deterministic solvers. The explicit use of convexity preservation under summation and the two-stage training procedure are clear strengths that distinguish the approach from standard compositional EBMs.

major comments (1)
  1. [Abstract] Abstract: The transfer claim (models trained on small subproblems transfer to larger instances) is load-bearing, yet the manuscript supplies no argument that the per-factor convex relaxation remains tight once the number of factors grows beyond the training scale. Convexity of the summed objective is immediate from the input-convex factors, but tightness of the relaxation (and quality of the projection step) is not shown to be preserved under increased composition width or depth; this directly affects whether relaxed optima remain useful for the original non-convex combinatorial task.
minor comments (1)
  1. [Abstract] The abstract refers to a 'tight convex relaxation' without indicating the specific construction or any verification procedure for tightness.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive review and for recognizing the potential of the CCEM framework. We address the single major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The transfer claim (models trained on small subproblems transfer to larger instances) is load-bearing, yet the manuscript supplies no argument that the per-factor convex relaxation remains tight once the number of factors grows beyond the training scale. Convexity of the summed objective is immediate from the input-convex factors, but tightness of the relaxation (and quality of the projection step) is not shown to be preserved under increased composition width or depth; this directly affects whether relaxed optima remain useful for the original non-convex combinatorial task.

    Authors: We agree that the manuscript does not provide a formal argument guaranteeing preservation of tightness under arbitrary increases in composition width or depth. Convexity of the objective follows immediately from summation of input-convex factors, but tightness of the per-factor relaxation and the quality of the subsequent projection are validated only empirically in the current experiments. The transfer results are therefore presented as an empirical observation rather than a general theoretical guarantee. In the revision we will add a short discussion paragraph clarifying this distinction, referencing the specific relaxation used for the feasible set and noting the empirical evidence that relaxed solutions remain useful on larger instances. We will also qualify the abstract to state that transfer is demonstrated experimentally. revision: yes

Circularity Check

0 steps flagged

No circularity; derivation rests on standard convexity properties and empirical transfer results

full rationale

The paper's central claims rely on the mathematical fact that convexity is preserved under summation (a standard property, not derived from the model's fitted parameters) and on two-stage training (contrastive learning followed by unrolled optimization) whose outputs are validated empirically on transfer to larger instances. No step reduces a prediction to a fitted input by construction, invokes a self-citation as the sole justification for a uniqueness claim, or renames a known result. The transfer claim is presented as an experimental outcome rather than a definitional consequence.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The approach rests on standard mathematical properties of convex functions and on the known architecture of input-convex neural networks; no new entities are postulated.

axioms (2)
  • standard math Convexity is preserved under summation of convex functions.
    Invoked to guarantee the composed objective remains convex.
  • domain assumption Input-convex neural networks produce convex energy functions with respect to their inputs.
    Taken from prior ICNN literature and used to shape each factor.

pith-pipeline@v0.9.1-grok · 5725 in / 1243 out tokens · 60342 ms · 2026-06-30T16:13:56.360901+00:00 · methodology

0 comments
read the original abstract

Compositional energy-based models can generalize to larger combinatorial reasoning problems by reusing a learned factor energy across many local constraints. In our paper, we show that a key bottleneck in compositional reasoning is not composition itself, but the non-convex geometry of the learned energy landscape. To solve this problem, we introduce Convex Compositional Energy Minimization (CCEM), a framework that parameterizes each factor with an input-convex neural network and optimizes the composed energy over a tight convex relaxation of the feasible set. Because convexity is preserved under summation, the global relaxed objective remains convex, enabling deterministic projected first-order optimization. CCEM is trained in two stages: factor-level contrastive learning to shape local energy basins, followed by end-to-end refinement through an unrolled projected solver. Our experiments show that our models trained on small subproblems or a single problem size transfer to larger instances without retraining.

Figures

Figures reproduced from arXiv: 2605.23395 by Abduragim Shtanchaev, Albert Baichorov, Arip Asadulaev, Dmitry V. Dylov, Fakhri Karray, Maksim Bobrin, Martin Tak\'a\v{c}, Meir Roketlishvili, Semyon Semenov, Viktor Kovalchuk.

Figure 1
Figure 1. Figure 1: Our method composes convex factors into a smooth globally convex landscape, enabling [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Energy map for 8-Queens. A correct solution is assigned low energy, while adding an extra queen produces higher local energy. We address four questions: (Q1) does enforcing convexity in the scope variable improve over the unconstrained MLP score network? (Q2) does projecting onto the exact convex hull of the feasible set replace the need for a long diffu￾sion chain? (Q3) how do projected (multi-start) Adam… view at source ↗
Figure 3
Figure 3. Figure 3: Optimized samples across projected Adam timesteps for 8-Queens. Intermediate columns [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Inference-start ablation for N-Queens. Increasing the number of independent starts [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Heatmap view of the inference-start ablation across board sizes. Larger boards require [PITH_FULL_IMAGE:figures/full_fig_p017_5.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 10 canonical work pages · 7 internal anchors

  1. [1]

    Input convex neural networks

    Brandon Amos, Lei Xu, and J Zico Kolter. Input convex neural networks. InInternational conference on machine learning, pages 146–155. PMLR, 2017

    2017

  2. [2]

    Connecting convex energy-based inference and optimal transport for domain adaptation

    Arip Asadulaev. Connecting convex energy-based inference and optimal transport for domain adaptation. InEnergy Based Models Workshop-ICLR 2021

    2021

  3. [3]

    A minimalist approach for domain adaptation with optimal transport

    Arip Asadulaev, Vitaly Shutov, Alexander Korotin, Alexander Panfilov, Vladislava Kontsevaya, and Andrey Filchenkov. A minimalist approach for domain adaptation with optimal transport. In Sarath Chandar, Razvan Pascanu, Hanie Sedghi, and Doina Precup, editors,Proceedings of The 2nd Conference on Lifelong Learning Agents, volume 232 ofProceedings of Machine...

    2023

  4. [4]

    Neural Combinatorial Optimization with Reinforcement Learning

    Irwan Bello, Hieu Pham, Quoc V Le, Mohammad Norouzi, and Samy Bengio. Neural combina- torial optimization with reinforcement learning.arXiv preprint arXiv:1611.09940, 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  5. [5]

    Optimal Control Via Neural Networks: A Convex Approach

    Yize Chen, Yuanyuan Shi, and Baosen Zhang. Optimal control via neural networks: A convex approach.arXiv preprint arXiv:1805.11835, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  6. [6]

    Xlvin: executed latent value iteration nets.arXiv preprint arXiv:2010.13146, 2020

    Andreea Deac, Petar Veliˇckovi´c, Ognjen Milinkovi´c, Pierre-Luc Bacon, Jian Tang, and Mladen Nikoli´c. Xlvin: executed latent value iteration nets.arXiv preprint arXiv:2010.13146, 2020

    work page arXiv 2010

  7. [7]

    Neural Turing Machines

    Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines.arXiv preprint arXiv:1410.5401, 2014

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  8. [8]

    Joshi, Thomas Laurent, and Xavier Bresson

    Chaitanya K Joshi, Thomas Laurent, and Xavier Bresson. An efficient graph convolutional network technique for the travelling salesman problem.arXiv preprint arXiv:1906.01227, 2019

    work page arXiv 1906

  9. [9]

    Neural GPUs Learn Algorithms

    Łukasz Kaiser and Ilya Sutskever. Neural gpus learn algorithms.arXiv preprint arXiv:1511.08228, 2015

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  10. [10]

    Semi-Supervised Classification with Graph Convolutional Networks

    Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907, 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  11. [11]

    Do neural optimal transport solvers work? a continuous wasserstein-2 benchmark.Advances in neural information processing systems, 34:14593–14605, 2021

    Alexander Korotin, Lingxiao Li, Aude Genevay, Justin M Solomon, Alexander Filippov, and Evgeny Burnaev. Do neural optimal transport solvers work? a continuous wasserstein-2 benchmark.Advances in neural information processing systems, 34:14593–14605, 2021

    2021

  12. [12]

    A tutorial on energy-based learning.Predicting structured data, 1(0), 2006

    Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, Fujie Huang, et al. A tutorial on energy-based learning.Predicting structured data, 1(0), 2006

    2006

  13. [13]

    Graph colouring meets deep learning: Effective graph neural network models for combinatorial problems

    Henrique Lemos, Marcelo Prates, Pedro Avelar, and Luis Lamb. Graph colouring meets deep learning: Effective graph neural network models for combinatorial problems. In2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), pages 879–885. IEEE, 2019

    2019

  14. [14]

    Fast t2t: Optimization consistency speeds up diffusion-based training-to-testing solving for combinatorial optimization

    Yang Li, Jinpei Guo, Runzhong Wang, Hongyuan Zha, and Junchi Yan. Fast t2t: Optimization consistency speeds up diffusion-based training-to-testing solving for combinatorial optimization. Advances in Neural Information Processing Systems, 37:30179–30206, 2024

    2024

  15. [15]

    Nsnet: A general neural probabilistic framework for satisfiability problems.Advances in Neural Information Processing Systems, 35:25573–25585, 2022

    Zhaoyu Li and Xujie Si. Nsnet: A general neural probabilistic framework for satisfiability problems.Advances in Neural Information Processing Systems, 35:25573–25585, 2022

    2022

  16. [16]

    Optimal transport mapping via input convex neural networks

    Ashok Makkuva, Amirhossein Taghvaei, Sewoong Oh, and Jason Lee. Optimal transport mapping via input convex neural networks. InInternational Conference on Machine Learning, pages 6672–6681. PMLR, 2020

    2020

  17. [17]

    Generalizable reasoning through compositional energy mini- mization.arXiv preprint arXiv:2510.20607, 2025

    Alexandru Oarga and Yilun Du. Generalizable reasoning through compositional energy mini- mization.arXiv preprint arXiv:2510.20607, 2025

    work page arXiv 2025

  18. [18]

    Can you learn an algorithm? generalizing from easy to hard problems with recurrent networks.Advances in Neural Information Processing Systems, 34:6695–6706, 2021

    Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, and Tom Goldstein. Can you learn an algorithm? generalizing from easy to hard problems with recurrent networks.Advances in Neural Information Processing Systems, 34:6695–6706, 2021. 11

    2021

  19. [19]

    Learning a SAT Solver from Single-Bit Supervision

    Daniel Selsam, Matthew Lamm, Benedikt Bünz, Percy Liang, Leonardo de Moura, and David L Dill. Learning a sat solver from single-bit supervision.arXiv preprint arXiv:1802.03685, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  20. [20]

    Difusco: Graph-based diffusion solvers for combinatorial optimization.Advances in neural information processing systems, 36:3706–3731, 2023

    Zhiqing Sun and Yiming Yang. Difusco: Graph-based diffusion solvers for combinatorial optimization.Advances in neural information processing systems, 36:3706–3731, 2023

    2023

  21. [21]

    Graph Attention Networks

    Petar Veliˇckovi´c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks.arXiv preprint arXiv:1710.10903, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  22. [22]

    Generative flow networks for discrete probabilistic modeling

    Dinghuai Zhang, Nikolay Malkin, Zhen Liu, Alexandra V olokhova, Aaron Courville, and Yoshua Bengio. Generative flow networks for discrete probabilistic modeling. InInternational Conference on Machine Learning, pages 26412–26428. PMLR, 2022. 12 A Proofs Proposition A.1(Non-convex factor composition admits spurious stable minima).Let Y ⊂R d be a convex rela...

    2022

  23. [23]

    The composed energy also contains a point¯y∈ Y, with¯y̸=y ⋆, satisfying ∇E(¯y) = 0,∇ 2E(¯y)≻0

  24. [24]

    Moreover, for gradient descent yt+1 =y t −η∇E(y t), there exists a stepsize η >0 and an open neighborhood U⊂ Y of ¯ysuch that every initialization y0 ∈U converges to ¯y

    Therefore,¯yis a strict local minimum ofE, but not a global minimizer: E(¯y)> E(y⋆). Moreover, for gradient descent yt+1 =y t −η∇E(y t), there exists a stepsize η >0 and an open neighborhood U⊂ Y of ¯ysuch that every initialization y0 ∈U converges to ¯y. Hence failure from such initializations is caused by the geometry of the composed energy landscape, ra...