pith. sign in

arxiv: 2606.31990 · v1 · pith:75UDHBTEnew · submitted 2026-06-30 · 💻 cs.NE · cs.LG

Evaluation of Population Initialization Methods for Genetic Programming-based Symbolic Regression

Lukas Kammerer , Gabriel Kronberger , Deaglan J. Bartlett , Harry Desmond , Pedro G. Ferreira , Stephan Winkler This is my paper

Pith reviewed 2026-07-01 02:00 UTC · model grok-4.3

classification 💻 cs.NE cs.LG
keywords genetic programmingsymbolic regressionpopulation initializationPareto optimizationNSGA-IIevolutionary algorithmsmodel complexity
0
0 comments X

The pith

The choice of initialization method has negligible impact on final results in genetic programming for symbolic regression when initial diversity is similar.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper compares three random initialization methods and one using small optimized solutions from exhaustive symbolic regression within a NSGA-II based genetic programming framework for symbolic regression. It evaluates the final Pareto fronts for accuracy and complexity on twelve synthetic problems of varying complexity and one real-world dataset. The results show no significant differences among the methods, with any initial advantage from the optimized initialization disappearing after only a few generations. This indicates that the initialization method does not substantially affect the outcome if the starting populations have similar diversity. A sympathetic reader would care because it means standard random initialization is sufficient for these tasks.

Core claim

We find no significant differences in accuracy or model complexity among the initialization methods. The initial advantage of initialization with ESR disappears after only a few generations. Our results show that, given similar diversity in the initial population, the effect of the initialization method in GP-based symbolic regression on the final Pareto front is negligible.

What carries the argument

Comparison of Pareto fronts produced by NSGA-II evolutionary search starting from different initial populations in genetic programming symbolic regression.

If this is right

  • The initial advantage of using optimized small solutions vanishes quickly during evolution.
  • Random initialization methods achieve comparable final accuracy and complexity trade-offs.
  • The effect of initialization is negligible when diversity levels are matched across methods.
  • Results are consistent across synthetic problems of different complexities and a real-world dataset.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Effort in symbolic regression via GP might be better spent on other parameters like selection or variation operators rather than initialization.
  • The finding could extend to other multi-objective evolutionary algorithms where population diversity is a key factor.
  • Further tests on a wider range of real-world datasets would help establish the generality of the negligible effect.

Load-bearing premise

The twelve synthetic problems and the one real-world dataset are representative of typical symbolic regression tasks.

What would settle it

Demonstrating a symbolic regression problem where one of the initialization methods produces a statistically superior Pareto front after the evolutionary process completes.

Figures

Figures reproduced from arXiv: 2606.31990 by Deaglan J. Bartlett, Gabriel Kronberger, Harry Desmond, Lukas Kammerer, Pedro G. Ferreira, Stephan Winkler.

Figure 1
Figure 1. Figure 1: We compare all initialization methods by running 1000 repetitions of GP with different random seeds on twelve synthetic and one real-world problem. We then com￾pare the resulting Pareto fronts for each problem [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Ground truth and 100 most accurate ESR solutions. In most problems, ESR finds models that capture the overall curvature of the ground truth but still have a high RMSE of between 10−2 and 10−1 compared to the noise level of σ = 10−7 . a population initialized with ESR provides sufficient diversity. To ensure that the subsequent evolutionary optimization is not compromised by a lack of diver￾sity, we compare… view at source ↗
Figure 3
Figure 3. Figure 3: The mean and standard deviation of pairwise tree distances among individuals in the initial population for each method. Higher values denote more diversity. As ESR initialization depends on the problem, the mean and standard deviation are shown for each problem separately. For the three random initialization methods, the bars show the mean and the average standard deviation of the initial populations of al… view at source ↗
Figure 4
Figure 4. Figure 4: The distribution of test errors of models in the initial population for all methods and problems. The dots show the median and the error bars denote the 5th and 95th percentile. The bars for the three random initialization methods show the average median and percentile values across all 1000 runs. As expected, we observe a much lower median test error in the ESR-initialized population than in the random in… view at source ↗
Figure 5
Figure 5. Figure 5: The distribution of test error over complexity of the Pareto-optimal models from each of the 1000 runs per initialization method for the synthetic problems. The areas between the dashed lines denote centered 90% of test errors and the solid line the median at a specific complexity. All methods show similar performance with very similar distributions denoted by largely overlapping areas between the dashed l… view at source ↗
Figure 6
Figure 6. Figure 6: The distribution of test error over complexity of the Pareto-optimal models from each of the 1000 runs for each initialization method in the synthetic Problem 11 after four different numbers of generations. While the ESR-initialized population performs better initially, these differences shrink continuously after ten and 20 generations until they perform equally after 200 generations. consistent with the o… view at source ↗
Figure 7
Figure 7. Figure 7: The distribution of test error over complexity of the Pareto-optimal models from each of the 1000 runs for each initialization method in the synthetic Problem 1 after four different numbers of generations. In contrast to all other problems, the ESR-initialized population shows a clear improvement over the randomly initialized population, which is due to the best ESR model being very close to the ground tru… view at source ↗
Figure 8
Figure 8. Figure 8: The one-dimensional real-world Nikuradse dataset [20, 9] and the GP results for each initialization method, in which we observe a similar pattern as in the synthetic problems. While ESR-initialization provides reasonable fits in the initial population, as shown in Figure 8a, the GP yields similar results over all initialization methods, as denoted by the overlapping error distributions in Figure 8b [PITH_… view at source ↗
Figure 9
Figure 9. Figure 9: The distribution of test error over complexity of the Pareto-optimal models from each of the 1000 runs for both methods in the Nikuradse dataset [20, 9] after four different numbers of generations. As in the synthetic problems, the advantage of the ESR-initialized population diminishes after only a few generations and does not lead to more accurate or less complex models. 4 Conclusion Despite the intuitive… view at source ↗
read the original abstract

We analyze the effect of optimizing the initial population of genetic programming (GP) for symbolic regression (SR) on the accuracy and complexity of solutions. We compare three well-established random initialization methods as well as initialization with small optimized solutions from exhaustive symbolic regression (ESR) using a GP/SR implementation which is based on the multi-objective evolutionary algorithm NSGA-II. We compare the final Pareto fronts found with each initialization method on twelve synthetic problems of varying complexity and one real-world dataset. We find no significant differences in accuracy or model complexity among the initialization methods. The initial advantage of initialization with ESR disappears after only a few generations. Our results show that, given similar diversity in the initial population, the effect of the initialization method in GP-based symbolic regression on the final Pareto front is negligible.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper empirically compares three random population initialization methods against initialization with small optimized solutions from exhaustive symbolic regression (ESR) in a multi-objective GP setup based on NSGA-II for symbolic regression. Experiments on twelve synthetic benchmarks of varying complexity and one real-world dataset show no significant differences in final Pareto fronts for accuracy and complexity; any initial ESR advantage vanishes after a few generations. The headline conclusion is that, given comparable initial diversity, the choice of initialization method has negligible impact on GP-SR outcomes.

Significance. If the diversity-equivalence premise and statistical claims hold, the result would indicate that GP-SR performance is robust to standard initialization choices once populations have comparable diversity, reducing the incentive to invest in specialized initialization routines and shifting attention to selection, variation, or objective design. The use of a reproducible multi-objective framework and a mix of synthetic plus real data strengthens the practical relevance of the finding.

major comments (2)
  1. [Abstract and §4] Abstract and §4 (Results): The central claim is explicitly conditional on 'given similar diversity in the initial population,' yet no diversity metrics (genotypic/phenotypic diversity, tree-size histograms, operator frequencies, or number of unique expressions) are reported at generation 0 to confirm that the ESR initialization produces statistically equivalent diversity to the three random methods. Without this verification the conditional conclusion does not follow from the experiments.
  2. [§3 and §4] §3 (Experimental Setup) and §4: The abstract asserts 'no significant differences' but the provided text supplies no information on the number of independent runs, the statistical tests employed, correction for multiple comparisons, p-value thresholds, or effect-size reporting. These details are required to evaluate whether the null result is powered and reliable.
minor comments (1)
  1. [Tables] Table captions and axis labels should explicitly state the diversity measure (if any) used to support the 'similar diversity' premise.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below.

read point-by-point responses

  1. Referee: [Abstract and §4] Abstract and §4 (Results): The central claim is explicitly conditional on 'given similar diversity in the initial population,' yet no diversity metrics (genotypic/phenotypic diversity, tree-size histograms, operator frequencies, or number of unique expressions) are reported at generation 0 to confirm that the ESR initialization produces statistically equivalent diversity to the three random methods. Without this verification the conditional conclusion does not follow from the experiments.

    Authors: We agree that explicit diversity metrics at generation 0 are required to substantiate the conditional claim. The revised manuscript will report phenotypic diversity (number of unique expressions) and tree-size distributions for the initial populations of all methods to allow verification of diversity equivalence. revision: yes

  2. Referee: [§3 and §4] §3 (Experimental Setup) and §4: The abstract asserts 'no significant differences' but the provided text supplies no information on the number of independent runs, the statistical tests employed, correction for multiple comparisons, p-value thresholds, or effect-size reporting. These details are required to evaluate whether the null result is powered and reliable.

    Authors: We agree that these statistical details are absent from the current text. The revised manuscript will expand §3 to include the number of independent runs, the statistical tests applied, corrections for multiple comparisons, p-value thresholds, and effect-size reporting. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical comparison with no derivations or self-referential steps

full rationale

The paper conducts direct experimental runs of NSGA-II-based GP on 12 synthetic benchmarks plus one real dataset, comparing four initialization methods and reporting final Pareto fronts. No equations, fitted parameters renamed as predictions, uniqueness theorems, or ansatzes appear. The central claim rests on observed outcomes after a few generations rather than any definitional reduction or self-citation chain. The 'given similar diversity' qualifier is an empirical precondition stated in the abstract but does not create circularity because diversity is an observable input property measured (or assumed) before the runs, not derived from the final result.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The study relies on standard assumptions from evolutionary computation and multi-objective optimization; no free parameters, new entities, or ad-hoc axioms are introduced beyond the choice of NSGA-II and the benchmark problems.

axioms (1)
  • domain assumption NSGA-II is an appropriate algorithm for multi-objective symbolic regression balancing accuracy and complexity.
    The paper states it uses an NSGA-II based GP/SR implementation.

pith-pipeline@v0.9.1-grok · 5674 in / 1208 out tokens · 45414 ms · 2026-07-01T02:00:37.880781+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    In: Proceedings of the Genetic and Evolutionary Com- putation Conference Companion

    Ahmad, H., Helmuth, T.: A comparison of semantic-based initialization methods for genetic programming. In: Proceedings of the Genetic and Evolutionary Com- putation Conference Companion. pp. 1878–1881 (2018)

    2018

  2. [2]

    Bartlett, D.J., Desmond, H., Ferreira, P.G.: Exhaustive symbolic regression func- tion sets (2022), https://doi.org/10.5281/zenodo.7339113

    work page doi:10.5281/zenodo.7339113 2022

  3. [3]

    IEEE Transactions on Evolutionary Computation28(4), 950–964 (2023)

    Bartlett, D.J., Desmond, H., Ferreira, P.G.: Exhaustive symbolic regression. IEEE Transactions on Evolutionary Computation28(4), 950–964 (2023)

    2023

  4. [4]

    In: Genetic and Evolutionary Computation Conference

    Burke, E., Gustafson, S., Kendall, G.: Ramped half-n-half initialisation bias in gp. In: Genetic and Evolutionary Computation Conference. pp. 1800–1801. Springer (2003)

    2003

  5. [5]

    In: Inter- national Conference on Computer Aided Systems Theory

    Burlacu, B., Kammerer, L., Affenzeller, M., Kronberger, G.: Hash-based tree simi- larity and simplification in genetic programming for symbolic regression. In: Inter- national Conference on Computer Aided Systems Theory. pp. 361–369. Springer (2020) Evaluation of Population Initialization Methods 15

    2020

  6. [6]

    In: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion

    Burlacu, B., Kronberger, G., Kommenda, M.: Operon C++ an efficient genetic pro- gramming framework for symbolic regression. In: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion. pp. 1562–1570 (2020)

    2020

  7. [7]

    IEEE transactions on evolutionary computation6(2), 182–197 (2002)

    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation6(2), 182–197 (2002)

    2002

  8. [8]

    Mathematical problems in engineering2015(1), 212794 (2015)

    Deng, Y., Liu, Y., Zhou, D.: An improved genetic algorithm with initial population strategy for symmetric tsp. Mathematical problems in engineering2015(1), 212794 (2015)

    2015

  9. [9]

    Science advances6(5), eaav6971 (2020)

    Guimerà, R., Reichardt, I., Aguilar-Mogas, A., Massucci, F.A., Miranda, M., Pal- larès, J., Sales-Pardo, M.: A bayesian machine scientist to aid in the solution of challenging scientific problems. Science advances6(5), eaav6971 (2020)

    2020

  10. [10]

    Information 9(7), 167 (2018)

    Hassanat, A.B., Prasath, V.S., Abbadi, M.A., Abu-Qdari, S.A., Faris, H.: An im- proved genetic algorithm with a new initialization mechanism based on regression techniques. Information 9(7), 167 (2018)

    2018

  11. [11]

    MIT Press (1992)

    Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press (1992)

    1992

  12. [12]

    CRC Press (2024)

    Kronberger, G., Burlacu, B., Kommenda, M., Winkler, S.M., Affenzeller, M.: Sym- bolic regression. CRC Press (2024)

    2024

  13. [13]

    Guiding Multi-Objective Genetic Programming with Description Length Improves Symbolic Regression Solutions

    Kronberger, G., de Franca, F.O., Bartlett, D.J., Desmond, H., Ferreira, P.G.: Guid- ingmulti-objectivegeneticprogrammingwithdescriptionlengthimprovessymbolic regression solutions. arXiv preprint arXiv:2605.22374 (2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  14. [14]

    In: Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks

    La Cava, W., Orzechowski, P., Burlacu, B., de França, F.O., Virgolin, M., Jin, Y., Kommenda, M., Moore, J.: Contemporary symbolic regression methods and their relative performance. In: Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. vol. 1 (2021)

    2021

  15. [15]

    In: Euro- pean Conference on Genetic Programming

    Langdon, W.B., Nordin, J.: Seeding genetic programming populations. In: Euro- pean Conference on Genetic Programming. pp. 304–315. Springer (2000)

    2000

  16. [16]

    Quarterly of applied mathematics2(2), 164–168 (1944)

    Levenberg, K.: A method for the solution of certain non-linear problems in least squares. Quarterly of applied mathematics2(2), 164–168 (1944)

    1944

  17. [17]

    IEEE Trans- actions on Evolutionary Computation4(3), 274–283 (2002)

    Luke, S.: Two fast tree-creation algorithms for genetic programming. IEEE Trans- actions on Evolutionary Computation4(3), 274–283 (2002)

    2002

  18. [18]

    Journal of the society for Industrial and Applied Mathematics11(2), 431–441 (1963)

    Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parame- ters. Journal of the society for Industrial and Applied Mathematics11(2), 431–441 (1963)

    1963

  19. [19]

    In: Proceedings of the 35th International Conference on Neural Information Processing Systems

    Mundhenk, T.N., Landajuela, M., Glatt, R., Santiago, C.P., Faissol, D.M., Pe- tersen, B.K.: Symbolic regression via neural-guided genetic programming popu- lation seeding. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. pp. 24912–24923 (2021)

    2021

  20. [20]

    VDI-Forschungsheft361, 1 (1933)

    Nikuradse, J.: Strömungsgesetze in rauhen Rohren. VDI-Forschungsheft361, 1 (1933)

    1933

  21. [21]

    Journal of Computational Science5(2), 277–297 (2014)

    Paul, P.V., Ramalingam, A., Baskaran, R., Dhavachelvan, P., Vivekanandan, K., Subramanian, R.: A new population seeding technique for permutation-coded ge- netic algorithm: Service transfer approach. Journal of Computational Science5(2), 277–297 (2014)

    2014