Full Nonlinear Velocity Reconstruction With Transformer and Ensemble Tree Machine Learning
Pith reviewed 2026-06-28 04:39 UTC · model grok-4.3
The pith
Machine learning models trained on simulation residuals recover nonlinear galaxy velocities more accurately than linear theory across wider scales.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Training both a gradient boosting decision tree and a Transformer to predict the residual between the actual velocity and the linear theory estimate for line-of-sight and transverse components allows the models to capture nonlinear contributions and outperform linear theory in velocity reconstruction on AbacusSummit mocks for DESI surveys.
What carries the argument
Residual prediction between true velocity and linear theory estimate, using multi-scale features fed to Transformer or gradient boosting tree models.
If this is right
- More accurate recovery of the velocity power spectrum than linear theory alone.
- Higher cross-correlation with true velocities maintained across a wider range of spatial scales.
- Improved estimates of cluster pairwise velocity correlations and stacked cluster density profiles for kSZ analyses.
Where Pith is reading between the lines
- The same residual-learning setup could be retrained on mocks that include different galaxy-halo prescriptions to test robustness before application to real catalogs.
- Velocity fields reconstructed this way might tighten constraints on modified gravity or dark energy when combined with existing density-field analyses.
Load-bearing premise
The AbacusSummit mocks accurately capture the nonlinear velocity field and survey systematics that will appear in real DESI and LSST data.
What would settle it
Training the models on one set of mocks and testing on an independent simulation suite or real DESI data yields no improvement over linear theory in power spectrum recovery or cross-correlation.
Figures
read the original abstract
Accurate reconstruction of peculiar velocities from galaxy positions is important for probing the motion and evolution of large scale structure. They are sensitive to the cosmological effects of gravity and dark sector matter, and complement other velocity inference methods such as the kinematic Sunyaev-Zel'dovich (kSZ) imprinted in the CMB. We show that machine learning methods improve velocity reconstruction by capturing nonlinear contributions. Specifically, we train both a gradient boosting decision tree (GBDT) and a Transformer using multi-scale features to predict the residual between the actual velocity and the estimate from linear theory for both the line-of-sight and transverse components. We evaluate our approach in both periodic box and, more realistic, lightcone settings using mock galaxy catalogs from the \textsc{AbacusSummit} simulations tailored to DESI spectroscopic surveys of luminous red galaxies (LRGs) and emission line galaxies (ELGs). We also assess the impact of redshift uncertainties such as those in Rubin LSST photometry. Both models significantly outperform the linear theory with the Transformer achieving the best performance. They more accurately recover the velocity power spectrum and maintain a higher cross-correlation with the true velocities across a wider range of spatial scales. Finally, we demonstrate two applications relevant to kSZ analyses: estimating cluster pairwise velocity correlations and stacked cluster density profiles. This machine learning framework for nonlinear velocity reconstruction opens up powerful new applications of survey data from DESI, Rubin LSST, Euclid and the Roman Space Telescope.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that gradient boosting decision trees (GBDT) and Transformer models, trained on multi-scale features extracted from AbacusSummit mocks tailored to DESI LRG/ELG selections, can predict nonlinear velocity residuals relative to linear theory. This yields improved recovery of the velocity power spectrum and higher cross-correlation coefficients with true velocities over a wider k-range in both periodic boxes and lightcones, plus a redshift-uncertainty test for LSST; two kSZ-related applications are also shown.
Significance. If the performance gains hold under broader validation, the approach could meaningfully extend the usable k-range for peculiar-velocity analyses in DESI, LSST, Euclid and Roman data. The work supplies a concrete, simulation-grounded demonstration that nonlinear ML corrections improve upon linear theory for both line-of-sight and transverse components, which is a useful incremental step for kSZ and velocity-field cosmology.
major comments (2)
- [Abstract] Abstract and evaluation sections: the reported outperformance on power-spectrum and cross-correlation metrics is stated without quantitative error bars, without ablation of the multi-scale feature set, and without explicit documentation of training/validation splits or hyperparameter selection procedures. These omissions make it impossible to assess whether the claimed gains are statistically robust or sensitive to modeling choices.
- [Evaluation on mocks] Evaluation on mocks and applications sections: no cross-simulation validation (e.g., on a second independent simulation suite with different baryonic physics or assembly bias) is presented. Because the central motivation is application to real DESI/LSST data, the absence of any test that the reported gains survive changes in the galaxy-halo connection or unmodeled systematics is load-bearing for the transfer claim.
minor comments (2)
- Notation for the residual velocity components and the precise definition of the multi-scale features should be consolidated into a single table or equation block for clarity.
- Figure captions for the power-spectrum and cross-correlation plots should include the exact k-range over which the improvement is quantified and the number of mocks used for the error estimation.
Simulated Author's Rebuttal
We thank the referee for their thoughtful comments, which have helped us identify areas to improve the clarity and robustness of our results. We address each major comment below.
read point-by-point responses
-
Referee: [Abstract] Abstract and evaluation sections: the reported outperformance on power-spectrum and cross-correlation metrics is stated without quantitative error bars, without ablation of the multi-scale feature set, and without explicit documentation of training/validation splits or hyperparameter selection procedures. These omissions make it impossible to assess whether the claimed gains are statistically robust or sensitive to modeling choices.
Authors: We agree with this assessment. The revised manuscript will include quantitative error bars on the power spectrum and cross-correlation metrics. We will also add an ablation study to evaluate the contribution of the multi-scale feature set and provide detailed documentation of the training/validation splits and hyperparameter optimization procedures in the methods section. revision: yes
-
Referee: [Evaluation on mocks] Evaluation on mocks and applications sections: no cross-simulation validation (e.g., on a second independent simulation suite with different baryonic physics or assembly bias) is presented. Because the central motivation is application to real DESI/LSST data, the absence of any test that the reported gains survive changes in the galaxy-halo connection or unmodeled systematics is load-bearing for the transfer claim.
Authors: We recognize that cross-simulation validation would provide stronger evidence for the applicability to real data. Our work utilizes the AbacusSummit simulations, and conducting validation on additional independent suites is beyond the scope of the current study due to computational constraints. In the revision, we will add a discussion of this limitation in the conclusions and suggest it as an important direction for future research. revision: partial
Circularity Check
No significant circularity; performance measured against external simulation truth
full rationale
The paper trains GBDT and Transformer models on multi-scale features from AbacusSummit mocks to predict velocity residuals relative to linear theory, then reports power-spectrum recovery and cross-correlation on held-out test mocks (both periodic boxes and lightcones) plus a redshift-uncertainty test. These metrics are computed against the simulation's independent ground-truth velocities, not by construction from the fitted model itself. No self-citation chain, ansatz smuggling, or renaming of known results forms the load-bearing step for the central empirical claims. The derivation is therefore self-contained against the stated simulation benchmarks.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Pairwise Velocity Statistic The pairwise velocity is a useful summary statistic of the cosmic velocity field, characterizing the average relative in- fall of tracer pairs as a function of their comoving separation. Following [73], the pairwise velocity estimator in a separation bin centered atris, V(r)= P i<j v∥,i −v ∥,j ci j P i<j c 2 i j .(15) The sum i...
-
[2]
displacement-only
Stacked kSZ profile We also consider the application of the reconstructed ve- locities to estimate a kSZ-derived halo electron density pro- file, a useful probe of the spatial distribution of ionized gas in galaxy clusters. This is obtained by stacking the kSZ temper- atures centered on cluster halos after weighting by their LOS velocities. In our analysi...
-
[3]
Simulated kSZ Map To assess how velocity reconstruction errors propagate into a practical kSZ analysis, we construct a mock kSZ temper- ature map from the halo lightcone catalog, following [80]. In the non-relativistic limit, the kSZ temperature fluctuation along the line of sight ˆncan be written as ∆TkS Z(ˆn) T0 =− σT c Z ne vlos dl,(A1) whereσ T is the...
-
[4]
Pairwise kSZ Momentum and Optical Depth Fitting The kSZ pairwise momentum is given by PkS Z(r,z i)=− P i j(∆ ¯TAP,i −∆ ¯TAP,j)ci j P i j c2 i j ,(A11) where the sum is over all pairs of tracers with separations in a bin centered onr, and in a redshift bin centered onz.∆ ¯TAP,i is the kSZ temperature obtained from simulated maps by using aperture photometr...
-
[5]
Kaiser, Mon
N. Kaiser, Mon. Not. Roy. Astron. Soc.227, 1 (1987)
1987
-
[6]
M. A. Strauss and J. A. Willick, Phys. Rep.261, 271 (1995), arXiv:astro-ph/9502079 [astro-ph]
work page internal anchor Pith review Pith/arXiv arXiv 1995
-
[7]
C. Howlett, K. Said, J. R. Lucey, M. Colless, F. Qin, Y . Lai, R. B. Tully, and T. M. Davis, Mon. Not. Roy. Astron. Soc.515, 953 (2022), arXiv:2201.03112 [astro-ph.CO]
-
[8]
W. J. Percival and M. White, Mon. Not. Roy. Astron. Soc.393, 297 (2009), arXiv:0808.0003 [astro-ph]
work page internal anchor Pith review Pith/arXiv arXiv 2009
-
[9]
M. J. Hudson and S. J. Turnbull, Astrophys. J. Lett.751, L30 (2013), arXiv:1203.4814 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2013
- [10]
- [11]
-
[12]
R. A. Sunyaev and Y . B. Zeldovich, Mon. Not. Roy. Astron. Soc.190, 413 (1980)
1980
-
[13]
Evidence of Galaxy Cluster Motions with the Kinematic Sunyaev-Zel'dovich Effect
N. Hand et al., Phys. Rev. Lett.109, 041101 (2012), arXiv:1203.4219 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2012
-
[14]
E. Schaan et al. (Atacama Cosmology Telescope), Phys. Rev. D 103, 063513 (2021), arXiv:2009.05557 [astro-ph.CO]
- [15]
-
[16]
Y . Gong et al., Phys. Rev. D113, 063538 (2026), arXiv:2511.23417 [astro-ph.CO]
-
[17]
C. M. Springob, C. Magoulas, M. Colless, J. Mould, P. Er- dogdu, D. H. Jones, J. R. Lucey, L. Campbell, and C. J. Fluke, Mon. Not. Roy. Astron. Soc.445, 2677 (2014), arXiv:1409.6161 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2014
- [18]
-
[19]
Future constraints on halo thermodynamics from combined Sunyaev-Zel'dovich measurements
N. Battaglia, S. Ferraro, E. Schaan, and D. Spergel, JCAP11, 040, arXiv:1705.05881 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv
-
[20]
D. J. Eisenstein, H.-j. Seo, and M. J. White, Astrophys. J.664, 660 (2007), arXiv:astro-ph/0604361
work page internal anchor Pith review Pith/arXiv arXiv 2007
-
[21]
D. J. Eisenstein, H.-j. Seo, E. Sirko, and D. Spergel, Astrophys. J.664, 675 (2007), arXiv:astro-ph/0604362
work page internal anchor Pith review Pith/arXiv arXiv 2007
-
[22]
N. Padmanabhan, X. Xu, D. J. Eisenstein, R. Scalzo, A. J. Cuesta, K. T. Mehta, and E. Kazin, Mon. Not. Roy. Astron. Soc. 427, 2132 (2012), arXiv:1202.0090 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2012
-
[23]
X. Xu, A. J. Cuesta, N. Padmanabhan, D. J. Eisenstein, and C. K. McBride, Mon. Not. Roy. Astron. Soc.431, 2834 (2013), arXiv:1206.6732 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2013
-
[24]
L. Anderson et al. (BOSS), Mon. Not. Roy. Astron. Soc.439, 83 (2014), arXiv:1303.4666 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[25]
H. Gil-Marin et al. (eBOSS), Mon. Not. Roy. Astron. Soc.498, 2492 (2020), arXiv:2007.08994 [astro-ph.CO]
- [26]
-
[27]
Reconstruction within the Zeldovich approximation
M. White, Mon. Not. Roy. Astron. Soc.450, 3822 (2015), arXiv:1504.03677 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2015
- [28]
-
[29]
E. Schaan et al. (ACTPol), Phys. Rev. D93, 082002 (2016), arXiv:1510.06442 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[30]
P. Zhang, J. Pan, and Y . Zheng, Physical Review D87, 10.1103/physrevd.87.063526 (2013)
-
[31]
C. Blake and R. J. Turner 10.33232/001c.124509 (2024), arXiv:2408.05660 [astro-ph.CO]
-
[32]
Y . Lai et al. 10.5281/zenodo.17602818 (2025), arXiv:2512.03229 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv doi:10.5281/zenodo.17602818 2025
-
[33]
The DESI Experiment Part I: Science,Targeting, and Survey Design
A. Aghamousa et al. (DESI), (2016), arXiv:1611.00036 [astro- ph.IM]
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[34]
LSST Science Collaboration, LSST Science Book, version 2.0 (2009), arXiv:0912.0201 [astro-ph.IM]
work page internal anchor Pith review Pith/arXiv arXiv 2009
-
[35]
Y . Mellier et al. (Euclid), Astron. Astrophys.697, A1 (2025), arXiv:2405.13491 [astro-ph.CO]
-
[36]
Wide-Field InfrarRed Survey Telescope-Astrophysics Focused Telescope Assets WFIRST-AFTA 2015 Report
D. Spergel et al., (2015), arXiv:1503.03757 [astro-ph.IM]
work page internal anchor Pith review Pith/arXiv arXiv 2015
- [37]
-
[38]
B. Hadzhiyska, S. Ferraro, B. R. Guachalla, and E. Schaan, Phys. Rev. D109, 103534 (2024), arXiv:2312.12434 [astro- ph.CO]
-
[39]
Y . B. Zel’dovich, Astron. Astrophys.5, 84 (1970)
1970
-
[40]
X. Xu, N. Padmanabhan, D. J. Eisenstein, K. T. Mehta, and A. J. Cuesta, Mon. Not. Roy. Astron. Soc.427, 2146 (2012), arXiv:1202.0091 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2012
-
[41]
Y .-Z. Ma, G.-D. Gong, N. Sui, and P. He, Mon. Not. Roy. As- tron. Soc.475, 379 (2018), arXiv:1711.08756 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2018
- [42]
- [43]
-
[44]
Suresh, H
R. Suresh, H. Bishnoi, A. V . Kuklin, A. Parikh, M. Molokeev, R. Harinarayanan, S. Gharat, and P. Hiba, Front. in Phys.12, 1322162 (2024)
2024
-
[45]
Y . Gong and R. Bean, Phys. Rev. D112, 023527 (2025), arXiv:2505.12720 [astro-ph.CO]
-
[46]
U-Net: Convolutional Networks for Biomedical Image Segmentation
O. Ronneberger, P. Fischer, and T. Brox, arXiv e-prints , arXiv:1505.04597 (2015), arXiv:1505.04597 [cs.CV]
work page internal anchor Pith review Pith/arXiv arXiv 2015
-
[47]
Y . Wang and X. Yang, Astrophys. J.969, 76 (2024), arXiv:2406.14101 [astro-ph.IM]
- [48]
-
[49]
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, arXiv e-prints , arXiv:2010.11929 (2020), arXiv:2010.11929 [cs.CV]
work page internal anchor Pith review Pith/arXiv arXiv 2010
-
[50]
Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, arXiv e-prints , arXiv:2103.14030 (2021), arXiv:2103.14030 [cs.CV]
work page internal anchor Pith review Pith/arXiv arXiv 2021
-
[51]
C. Li, Y . Zhang, C. Cui, D. Fan, Y . Zhao, X.-B. Wu, B. He, Y . Xu, S. Li, J. Han, Y . Tao, L. Mi, H. Yang, and S. Yang, Monthly Notices of the Royal Astronomical Society 10.1093/mnras/stab1650 (2021)
-
[52]
N. Sahakyan, V . Vardanyan, and M. Khachatryan, Mon. Not. Roy. Astron. Soc.519, 3000 (2022), arXiv:2212.06614 [astro- ph.HE]
-
[53]
A. Tolamatti, K. K. Singh, and K. K. Yadav, Mon. Not. Roy. As- tron. Soc.523, 5341 (2023), arXiv:2306.15393 [astro-ph.HE]
-
[54]
J. Coronado-Blázquez, Mon. Not. Roy. Astron. Soc.521, 4156 (2023), arXiv:2303.07922 [astro-ph.HE]
-
[55]
Donoso-Oliva, I
C. Donoso-Oliva, I. Becker, P. Protopapas, G. Cabrera-Vives, M. Vishnu, and H. Vardhan, Astron. Astrophys.670, A54 (2023)
2023
-
[56]
H. W. Leung and J. Bovy, Monthly Notices of the Royal Astro- nomical Society527, 1494–1520 (2023)
2023
-
[57]
Y . Gong and R. Bean, Phys. Rev. D109, 123525 (2024), arXiv:2403.04664 [astro-ph.CO]
-
[58]
T. Signor et al. (Euclid), (2024), arXiv:2402.04800 [astro- 19 ph.CO]
-
[59]
J. H. Friedman, The Annals of Statistics29, 1189 (2001)
2001
-
[60]
G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y . Liu, in Advances in Neural Information Processing Systems, V ol. 30 (2017)
2017
-
[61]
Vaswani, N
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, in Advances in Neural Information Processing Systems, V ol. 30 (2017)
2017
- [62]
-
[63]
L. H. Garrison, D. J. Eisenstein, and P. A. Pinto, Mon. Not. Roy. Astron. Soc.485, 3370 (2019), arXiv:1810.02916 [astro- ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2019
- [64]
-
[65]
Planck 2018 results. VI. Cosmological parameters
N. Aghanim et al. (Planck), Astron. Astrophys.641, A6 (2020), [Erratum: Astron.Astrophys. 652, C4 (2021)], arXiv:1807.06209 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2020
- [66]
-
[67]
S. Yuan et al., Mon. Not. Roy. Astron. Soc.530, 947 (2024), arXiv:2306.06314 [astro-ph.CO]
-
[68]
Galaxy Evolution from Halo Occupation Distribution Modeling of DEEP2 and SDSS Galaxy Clustering
Z. Zheng, A. L. Coil, and I. Zehavi, Astrophys. J.667, 760 (2007), arXiv:astro-ph/0703457
work page internal anchor Pith review Pith/arXiv arXiv 2007
-
[69]
Hadzhiyska et al., (2023), arXiv:2305.11935 [astro-ph.CO]
B. Hadzhiyska et al., (2023), arXiv:2305.11935 [astro-ph.CO]
-
[70]
J. L. Tinker, B. E. Robertson, A. V . Kravtsov, A. Klypin, M. S. Warren, G. Yepes, and S. Gottlöber, Astrophys. J.724, 878 (2010), arXiv:1001.3162 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2010
-
[71]
B. Hadzhiyska, L. H. Garrison, D. Eisenstein, and S. Bose, Mon. Not. Roy. Astron. Soc.509, 2194 (2021), arXiv:2110.11413 [astro-ph.CO]
-
[72]
V . Desjacques, D. Jeong, and F. Schmidt, Phys. Rept.733, 1 (2018), arXiv:1611.09787 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2018
- [73]
- [74]
-
[75]
B. Soergel, A. Saro, T. Giannantonio, G. Efstathiou, and K. Dolag, Mon. Not. Roy. Astron. Soc.478, 5320 (2018), arXiv:1712.05714 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[76]
B. Hadzhiyska, S. Ferraro, R. Pakmor, S. Bose, A. M. Delgado, C. Hernández-Aguayo, R. Kannan, V . Springel, S. D. M. White, and L. Hernquist, Mon. Not. Roy. Astron. Soc.526, 369 (2023), arXiv:2305.00992 [astro-ph.CO]
- [77]
-
[78]
J. E. Forero-Romero, Y . Hoffman, S. Gottloeber, A. Klypin, and G. Yepes, Mon. Not. Roy. Astron. Soc.396, 1815 (2009), arXiv:0809.4135 [astro-ph]
work page internal anchor Pith review Pith/arXiv arXiv 2009
-
[79]
L. Lucie-Smith, H. V . Peiris, and A. Pontzen, Mon. Not. Roy. Astron. Soc.490, 331 (2019), arXiv:1906.06339 [astro-ph.CO]
-
[80]
Designing a space-based galaxy redshift survey to probe dark energy
Y . Wanget al., Mon. Not. Roy. Astron. Soc.409, 737 (2010), arXiv:1006.3517 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv 2010
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.