Pith. sign in

REVIEW 2 major objections 2 minor 300 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

Reasoning language models show large adoption gaps across 28 scientific disciplines, widest outside hard sciences.

2026-07-01 07:20 UTC pith:6UR3WSND

load-bearing objection A broad ERC-based survey of RLM resources that flags gaps in social sciences but counts benchmarks instead of measuring uptake or impact. the 2 major comments →

arxiv 2606.01145 v3 pith:6UR3WSND submitted 2026-05-31 cs.AI

Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branches

Teddy Ferdinan , Bart{\l}omiej Koptyra , Miko{\l}aj Langner , Tomasz Adamczyk , {\L}ukasz Radli\'nski , Maciej Markiewicz , Aleksander Szcz\k{e}sny , Stanis{\l}aw Wo\'zniak
show 37 more authors
Tymoteusz Romanowicz Dzmitry Pihulski Mateusz Zbrocki Mateusz \'Smigielski Micha{\l} Rajkowski Mateusz Biedka Konrad Kie{\l}czy\'nski Konrad Wojtasik Jacek Duszenko Jan Eliasz Piotr Matys Micha{\l} Bernacki-Janson Maria Bellaniar Ismiati Latius Hermawan Wiktoria Mieleszczenko-Kowszewicz Anna Kubicka-Sowi\'nska Grzegorz Chodak Karol Postawa Pawe{\l} Zyblewski Tomasz Szanda{\l}a {\L}ukasz Sterczewski Adrian Chajec Pawe{\l} Niewiadomski Piotr Gruber Marcin Wdowikowski S{\l}awomir Czarnecki Bart{\l}omiej Kryszak Dominik Drabik Tomasz Kajdanowicz Kamil Mamak Pawe{\l} Pre\'s Katarzyna Paczkowska Joachim Sobczuk Tomasz Zi\k{e}ba Jan Koco\'n Maciej Piasecki Przemys{\l}aw Kazienko
This is my paper
classification cs.AI
keywords reasoning language modelsscientific disciplinesERC classificationmaturity assessmentadoption disparitiessurveyresource availability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper surveys how reasoning language models are developed, evaluated, and used in every ERC-defined field from social sciences and humanities through physical sciences to life sciences. It introduces a maturity framework that scores disciplines on the presence of domain-specific training data, benchmarks, and applications. The survey finds hard-science fields far ahead while others lag, and the gap widens sharply when only publicly available resources are counted. If the measured differences are real, research productivity will continue to diverge unless resources are deliberately built for the lagging fields.

Core claim

RLMs are concentrated in hard-science disciplines; a maturity-oriented assessment of development and evaluation resources across the 28 ERC fields shows substantial disparities that become larger when restricted to public resources.

What carries the argument

maturity-oriented assessment framework that scores disciplines by the quantity and quality of domain-specific development and evaluation resources

Load-bearing premise

The ERC discipline labels and the authors' resource-based maturity scores correctly track genuine differences in how ready each field is for RLMs.

What would settle it

A new inventory that finds comparable numbers of public RLM training sets, benchmarks, and deployed applications in social-science and humanities fields as in physics and engineering would falsify the reported disparities.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Implementation patterns that succeed in hard sciences can be examined for transfer to other domains.
  • Public-resource scarcity in lagging fields limits reproducibility and slows progress.
  • Current challenges include lack of domain data and evaluation standards outside core sciences.
  • Future directions center on building shared resources to close the maturity gaps.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Targeted public datasets and benchmarks for social sciences and humanities could narrow the productivity gap faster than general-purpose models alone.
  • If maturity scores correlate with actual research output gains, funders could use the framework to prioritize resource creation in low-maturity fields.
  • The survey's focus on public resources implies that closed or proprietary RLMs may be masking even larger field differences.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper presents a literature survey claiming to be the first comprehensive analysis of Reasoning Language Models (RLMs) adoption across 28 scientific disciplines using the ERC classification. It introduces a maturity-oriented assessment framework based on the presence of domain-specific development and evaluation resources (public vs. all), reports substantial disparities in RLM maturity that widen when restricting to public resources, and discusses implementation paradigms, challenges, and future directions for broader adoption.

Significance. If the survey methodology is rigorous and the maturity framework is shown to correlate with real-world adoption, the work could usefully map gaps in RLM use across sciences and motivate targeted resource development. The explicit coverage of all 28 ERC panels and the public-vs-all distinction are concrete contributions that could inform funding and benchmark creation priorities.

major comments (2)
  1. [maturity-oriented assessment framework] The maturity-oriented assessment framework (introduced after the ERC classification discussion) scores disciplines solely by counts of domain-specific resources without demonstrating correlation to downstream adoption indicators such as citation-weighted RLM usage, researcher surveys, or measured productivity changes. This makes the central claim of 'substantial disparities in RLM maturity' rest on an unvalidated proxy and risks conflating resource availability with actual scientific uptake.
  2. [survey methodology] No section provides the search methodology, inclusion/exclusion criteria, database sources, or counting protocol used to identify and tally the 'development and evaluation resources' across the 28 disciplines. Without these details the reported disparities cannot be reproduced or verified, directly affecting the soundness of the headline finding.
minor comments (2)
  1. [Abstract] The abstract states the work spans 'Social Sciences and Humanities, Physical Sciences and Engineering, and Life Sciences' but does not list the exact 28 ERC panels or provide a table mapping them; adding this would improve traceability.
  2. [Abstract] Notation for 'RLM' is introduced without an explicit definition on first use in the abstract; a parenthetical expansion would aid readers outside the immediate subfield.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We respond to each major comment below, providing clarifications and indicating where the manuscript will be revised for improved rigor and transparency.

read point-by-point responses

  1. Referee: The maturity-oriented assessment framework (introduced after the ERC classification discussion) scores disciplines solely by counts of domain-specific resources without demonstrating correlation to downstream adoption indicators such as citation-weighted RLM usage, researcher surveys, or measured productivity changes. This makes the central claim of 'substantial disparities in RLM maturity' rest on an unvalidated proxy and risks conflating resource availability with actual scientific uptake.

    Authors: We acknowledge that the framework employs resource counts as a proxy for maturity and does not include direct validation against adoption metrics such as citation counts or researcher surveys. This proxy was selected to permit a uniform assessment across all 28 ERC disciplines, where granular adoption data are not consistently available in the literature. In revision we will explicitly label the measure as a resource-based proxy, discuss its limitations, and note that future work could correlate it with productivity indicators. The reported disparities will be reframed as differences in resource availability rather than proven uptake. revision: partial

  2. Referee: No section provides the search methodology, inclusion/exclusion criteria, database sources, or counting protocol used to identify and tally the 'development and evaluation resources' across the 28 disciplines. Without these details the reported disparities cannot be reproduced or verified, directly affecting the soundness of the headline finding.

    Authors: The current version of the manuscript does not contain a dedicated methods section. We will add one that specifies the databases searched (arXiv, Google Scholar, PubMed, and discipline-specific repositories), the Boolean queries used, inclusion and exclusion criteria for papers and resources, and the exact counting protocol applied to development and evaluation resources. This addition will make the survey fully reproducible. revision: yes

Circularity Check

0 steps flagged

No circularity: survey applies explicitly defined framework to external literature counts

full rationale

This is a literature survey paper with no mathematical derivations, equations, fitted parameters, or predictive models. The maturity framework is introduced as an author-defined classification scheme based on counting domain-specific resources (development and evaluation), then applied to ERC disciplines using external sources. No step reduces a central claim to a self-citation chain, self-definition, or fitted input renamed as prediction; the analysis remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The survey rests on the domain assumption that the ERC taxonomy is a neutral and complete partition of science and that counting 'domain-specific development and evaluation resources' is a valid proxy for RLM maturity. No free parameters or invented entities are introduced.

axioms (2)
  • domain assumption The European Research Council classification provides an appropriate and exhaustive partitioning of scientific disciplines for this analysis.
    Invoked in the abstract when stating the scope of 28 disciplines.
  • domain assumption Maturity can be meaningfully ranked by the presence of domain-specific development and evaluation resources.
    Central to the introduced assessment framework.

pith-pipeline@v0.9.1-grok · 5990 in / 1245 out tokens · 27814 ms · 2026-07-01T07:20:22.713961+00:00 · methodology

0 comments
read the original abstract

While Reasoning Language Models (RLMs) are rapidly emerging as powerful tools for scientific research, their impact is primarily concentrated in "hard science" fields. The slow -- or lack of -- adoption of RLMs in other branches of science is causing a widening gap in research productivity. In this survey, we provide the first comprehensive analysis of RLM adoption across 28 scientific disciplines following the classification used by the European Research Council (ERC), spanning the Social Sciences and Humanities, Physical Sciences and Engineering, and Life Sciences. We examine how RLMs are developed, evaluated, and applied across disciplines. Furthermore, we introduce a maturity-oriented assessment framework based on available domain-specific development and evaluation resources, revealing substantial disparities in RLM maturity that become even more pronounced when only publicly available resources are considered. Finally, we highlight current implementation paradigms that are gaining popularity across disciplines, current challenges, and future directions in enabling RLM adoption across science.

Figures

Figures reproduced from arXiv: 2606.01145 by Adrian Chajec, Aleksander Szcz\k{e}sny, Anna Kubicka-Sowi\'nska, Bart{\l}omiej Koptyra, Bart{\l}omiej Kryszak, Dominik Drabik, Dzmitry Pihulski, Grzegorz Chodak, Jacek Duszenko, Jan Eliasz, Jan Koco\'n, Joachim Sobczuk, Kamil Mamak, Karol Postawa, Katarzyna Paczkowska, Konrad Kie{\l}czy\'nski, Konrad Wojtasik, Latius Hermawan, {\L}ukasz Radli\'nski, {\L}ukasz Sterczewski, Maciej Markiewicz, Maciej Piasecki, Marcin Wdowikowski, Maria Bellaniar Ismiati, Mateusz Biedka, Mateusz \'Smigielski, Mateusz Zbrocki, Micha{\l} Bernacki-Janson, Micha{\l} Rajkowski, Miko{\l}aj Langner, Pawe{\l} Niewiadomski, Pawe{\l} Pre\'s, Pawe{\l} Zyblewski, Piotr Gruber, Piotr Matys, Przemys{\l}aw Kazienko, S{\l}awomir Czarnecki, Stanis{\l}aw Wo\'zniak, Teddy Ferdinan, Tomasz Adamczyk, Tomasz Kajdanowicz, Tomasz Szanda{\l}a, Tomasz Zi\k{e}ba, Tymoteusz Romanowicz, Wiktoria Mieleszczenko-Kowszewicz.

Figure 1
Figure 1. Figure 1: RLM maturity levels across 28 scientific disciplines based on all identified resources. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Maturity of Development Resources (MD) and Maturity of Evaluation Resources (ME) of three main research domains: Social [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 2
Figure 2. Figure 2: Maturity of Development Resources (MD) and Maturity of Evaluation Resources (ME) for 28 scientific disciplines as calculated [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Maturity of Development Resources (MD) and Maturity of Evaluation Resources (ME) for 28 scientific disciplines as calculated [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 3
Figure 3. Figure 3: Maturity of Development Resources (MD) and Maturity of Evaluation Resources (ME) of three main research domains: Social [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Current trends of RLM usage in science: (a) Specialized models over general, all-purpose models; (b) Explainability via reasoning [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Current challenges of RLM adoption in science: (a) Hallucination & unfaithful reasoning; (b) Limited interpolative analysis; (c) [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Future directions for RLM adoption in science: (a) Grounding on multimodal scientific data; (b) Management of scientific [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 6
Figure 6. Figure 6: Future directions for RLM adoption in science: (a) Grounding on multimodal scientific data; (b) Management of scientific [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

300 extracted references · 217 canonical work pages · 25 internal anchors

  1. [1]

    Moore, Daniel M

    Suhaib Abdurahman, Alireza Salkhordeh Ziabari, Alexander K. Moore, Daniel M. Bartels, and Morteza Dehghani. 2025. A Primer for Evaluating Large Language Models in Social-Science Research.Advances in Methods and Practices in Psychological Science8, 2 (2025), 25152459251325174. arXiv:https://doi.org/10.1177/25152459251325174 doi:10.1177/25152459251325174 Re...

    work page doi:10.1177/25152459251325174 2025

  2. [2]

    Muntasir Adnan and Carlos C. N. Kuhn. 2025. Measuring and mitigating debugging effectiveness decay in code language models.Scientific Reports 15, 1 (18 Dec 2025), 44120. doi:10.1038/s41598-025-27846-5

    work page doi:10.1038/s41598-025-27846-5 2025

  3. [3]

    Said Al Faraby, Ade Romadhony, and Adiwijaya. 2024. Analysis of LLMs for educational question classification and generation.Computers and Education: Artificial Intelligence7 (2024), 100298. doi:10.1016/j.caeai.2024.100298

    work page doi:10.1016/j.caeai.2024.100298 2024

  4. [4]

    Nourah Alangari and Nouf AlShenaifi. 2025. N&N at QIAS 2025: Chain-of-Thought Ensembles with Retrieval-Augmented framework for Classical Arabic Islamic. InProceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks, Kareem Darwish, Ahmed Ali, Ibrahim Abu Farha, Samia Touileb, Imed Zitouni, Ahmed Abdelali, Sharefah Al-Ghamdi, Sakha...

    work page doi:10.18653/v1/2025.arabicnlp-sharedtasks.121 2025

  5. [5]

    Asma Musabah Alkalbani, Ahmed Salim Alrawahi, Ahmad Salah, Venus Haghighi, Yang Zhang, Salam Alkindi, and Quan Z. Sheng. 2025. A Systematic Review of Large Language Models in Medical Specialties: Applications, Challenges and Future Directions.Information16, 6 (2025). doi:10.3390/info16060489

    work page doi:10.3390/info16060489 2025

  6. [6]

    Santiago Alonso Sousa, Syed Saad Ul Hassan Bukhari, Paulo Vinicius Steagall, Paweł M Bęczkowski, Antonio Giuliano, and Kate J Flay. 2025. Performance of large language models on veterinary undergraduate multiple-choice examinations: a comparative evaluation.Front Vet Sci12 (Aug. 2025), 1616566

    2025

  7. [7]

    Maryam Amirizaniani, Elias Martin, Maryna Sivachenko, Afra Mashhadi, and Chirag Shah. 2024. Can LLMs Reason Like Humans? Assessing Theory of Mind Reasoning in LLMs for Open-Ended Questions. InProceedings of the 33rd ACM International Conference on Information and Knowledge Management(Boise, ID, USA)(CIKM ’24). Association for Computing Machinery, New York...

    work page doi:10.1145/3627673.3679832 2024

  8. [8]

    Richardson, Austin C

    Jacy Reese Anthis, Ryan Liu, Sean M. Richardson, Austin C. Kozlowski, Bernard Koch, James Evans, Erik Brynjolfsson, and Michael Bernstein

  9. [9]

    Llm social simulations are a promising research method.arXiv preprint arXiv:2504.02234,

    LLM Social Simulations Are a Promising Research Method. arXiv:2504.02234 [cs.HC] https://arxiv.org/abs/2504.02234

    work page arXiv

  10. [10]

    2026.System Card: Claude Opus 4.6

    Anthropic. 2026.System Card: Claude Opus 4.6. Technical Report. Anthropic. https://www-cdn.anthropic.com/ 14e4fb01875d2a69f646fa5e574dea2b1c0ff7b5.pdf Accessed: 2026-04-14

    2026

  11. [11]

    Antunes, Keith T

    Luis M. Antunes, Keith T. Butler, and Ricardo Grau-Crespo. 2024. Crystal structure generation with autoregressive large language modeling. Nature Communications15, 1 (06 Dec 2024), 10570. doi:10.1038/s41467-024-54639-7

    work page doi:10.1038/s41467-024-54639-7 2024

  12. [12]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. InThe Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=hSyW5go0v8

    2024

  13. [13]

    Muhammad Ayoub, Hai Zhao, Lifeng Li, Dongjie Yang, Shabir Hussain, and Junaid Abdul Wahid. 2026. Structured clinical approach to enable large language models to be used for improved clinical diagnosis and explainable reasoning.Communications Medicine6, 1 (Jan. 2026), 86. doi:10.1038/s43856-025-01348-x

    work page doi:10.1038/s43856-025-01348-x 2026

  14. [14]

    Chaithanya Bandi, Ben Hertzberg, Geobio Boo, Tejas Polakam, Jeff Da, Sami Hassaan, Manasi Sharma, Andrew Park, Ernesto Hernandez, Dan Rambado, Ivan Salazar, Rafael Cruz, Chetan Rane, Ben Levin, Brad Kenstler, and Bing Liu. 2026. MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers. arXiv:2602.00933 [cs.SE] https://arxiv.org/abs...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  15. [15]

    Barman, Sascha Caron, Emily Sullivan, Henk W

    Kristian G. Barman, Sascha Caron, Emily Sullivan, Henk W. de Regt, Roberto Ruiz de Austri, Mieke Boon, Michael Färber, Stefan Fröse, Tobias Golling, Luis G. Lopez, Faegheh Hasibi, Lukas Heinrich, Andreas Ipp, Rukshak Kapoor, Gregor Kasieczka, Daniel Kostić, Michael Krämer, Jesus Marco, Sydney Otten, Pawel Pawlowski, Pietro Vischia, Erik Weber, and Christo...

    work page doi:10.1140/epjc/s10052-025-14707-8 2025

  16. [16]

    Leo Benac, Jonas Raedler, Zilin Ma, and Finale Doshi-Velez. 2026. A Benchmark for Multi-Party Negotiation Games from Real Negotiation Data. arXiv:2603.14066 [cs.MA] https://arxiv.org/abs/2603.14066

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  17. [17]

    Armin Berger, Sarthak Khanna, David Berghaus, and Rafet Sifa. 2025. Reasoning LLMs in the Medical Domain: A Literature Survey.arXiv preprint arXiv:2508.19097(2025)

    work page arXiv 2025

  18. [18]

    Montine, and Nima Aghaeepour

    Eloise Berson, Philip Chung, Camilo Espinosa, Thomas J. Montine, and Nima Aghaeepour. 2024. Unlocking human immune system complexity through AI.Nature Methods21 (2024), 1400–1402. doi:10.1038/s41592-024-02351-1

    work page doi:10.1038/s41592-024-02351-1 2024

  19. [19]

    Randolph, Matthew Muhoberac, Palak Manchanda, Katherine A

    Connor Beveridge, Sanjay Iyer, Caitlin E. Randolph, Matthew Muhoberac, Palak Manchanda, Katherine A. Walker, Shane Tichy, and Gaurav Chopra. 2025. CLAW-MRM: Comprehensive Lipidomics Automation Workflow for Multiple Reaction Monitoring Using Large Language Models. Analytical Chemistry97, 36 (2025), 19409–19418. arXiv:https://doi.org/10.1021/acs.analchem.4c...

    work page doi:10.1021/acs.analchem.4c05039 2025

  20. [20]

    Nathan Bigaud, Vincent Cabeli, Meltem Gürel, Arthur Pignet, John Klein, Gilles Wainrib, and Eric Durand. 2025. OwkinZero: Accelerating Biological Discovery with AI. arXiv:2508.16315 [cs.LG] https://arxiv.org/abs/2508.16315

    work page arXiv 2025

  21. [21]

    Francisco CJ Bonfim, Sara Pessoa Silva, Alicia S Neves, Henrique Santos, João A Monteiro Neto, Jorge LB Araujo, Rilder S Pires, Ricardo Alexandre Da Silva Costa, Vasco Furtado, and Vladia Pinheiro. 2026. SARA: Leveraging LLM Agents and Jurisprudential Ontologies for Automated Legal Reasoning. InProceedings of the AAAI Conference on Artificial Intelligence...

    2026

  22. [22]

    Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D

    Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D. White, and Philippe Schwaller. 2024. Augmenting large language models with chemistry tools.Nature Machine Intelligence(2024). doi:10.1038/s42256-024-00832-8

    work page doi:10.1038/s42256-024-00832-8 2024

  23. [23]

    Fan Bu et al. 2025. An Investigation into Value Misalignment in LLM-Generated Texts for Cultural Heritage.arXiv preprint arXiv:2501.02039(2025). https://arxiv.org/abs/2501.02039 18 Ferdinan et al

    work page arXiv 2025

  24. [24]

    Spears, Derya Unutmaz, Kevin Weil, Steven Yin, and Nikita Zhivotovskiy

    Sébastien Bubeck, Christian Coester, Ronen Eldan, Timothy Gowers, Yin Tat Lee, Alexandru Lupsasca, Mehtaab Sawhney, Robert Scherrer, Mark Sellke, Brian K. Spears, Derya Unutmaz, Kevin Weil, Steven Yin, and Nikita Zhivotovskiy. 2025. Early science acceleration experiments with GPT-5. arXiv:2511.16072 [cs.CL] https://arxiv.org/abs/2511.16072

    work page arXiv 2025

  25. [25]

    Hua Cai, Shuang Zhao, Liang Zhang, Xuli Shen, Qing Xu, Weilin Shen, Zihao Wen, and Tianke Ban. 2025. Unilaw-R1: A Large Language Model for Legal Reasoning with Reinforcement Learning and Iterative Inference. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Ro...

    work page doi:10.18653/v1/2025.emnlp-main.915 2025

  26. [26]

    Quintina Campbell, Sam Cox, Jorge Medina, Brittany Watterson, and Andrew D White. 2025. MDCrow: Automating Molecular Dynamics Workflows with Large Language Models.arXiv preprint arXiv:2502.09565(2025)

    work page arXiv 2025

  27. [27]

    Cardille, Renee Johnston, Simon Ilyushchenko, et al

    Jeffrey A. Cardille, Renee Johnston, Simon Ilyushchenko, et al. 2025. The Cloud-Based Geospatial Benchmark: Challenges and LLM Evaluation. In Proceedings of The TerraBytes ICML Workshop: Towards global datasets and models for Earth Observation (Proceedings of Machine Learning Research, Vol. 292), Nicolas Audebert, Hossein Azizpour, Valentin Barrière, Javi...

    2025

  28. [28]

    Alejandro Carrasco, Victor Rodriguez-Fernandez, and Richard Linares. 2025. Large language models as autonomous spacecraft operators in kerbal space program.Advances in Space Research76, 6 (2025), 3480–3497. doi:10.1016/j.asr.2025.06.034

    work page doi:10.1016/j.asr.2025.06.034 2025

  29. [29]

    Santiago Casas, Christian Fidler, Boris Bolliet, Francisco Villaescusa-Navarro, and Julien Lesgourgues. 2025. CLAPP: The CLASS LLM Agent for Pair Programming. arXiv:2508.05728 [astro-ph.IM] https://arxiv.org/abs/2508.05728

    work page arXiv 2025

  30. [30]

    Harry Caufield, Harshad Hegde, Vincent Emonet, et al

    J. Harry Caufield, Harshad Hegde, Vincent Emonet, et al. 2024. Structured Prompt Interrogation and Recursive Extraction of Semantics (SPIRES): a method for populating knowledge bases using zero-shot learning.Bioinformatics40, 3 (2024), btae104. doi:10.1093/bioinformatics/btae104

    work page doi:10.1093/bioinformatics/btae104 2024

  31. [31]

    Ajenifujah, Janghoon Ock, and Amir Barati Farimani

    Achuth Chandrasekhara, Omid Barati Farimani, Olabode T. Ajenifujah, Janghoon Ock, and Amir Barati Farimani. 2025. NANOGPT: A Query- Driven Large Language Model Retrieval-Augmented Generation System for Nanotechnology Research.arXiv preprint arXiv:2502.20541(2025). doi:10.48550/arXiv.2502.20541

    work page doi:10.48550/arxiv.2502.20541 2025

  32. [32]

    Sreejato Chatterjee, Linh Tran, Quoc Duy Nguyen, Roni Kirson, Drue Hamlin, Harvest Aquino, Hanjia Lyu, Jiebo Luo, and Timothy Dye. 2025. Assessing Historical Structural Oppression Worldwide via Rule-Guided Prompting of Large Language Models.arXiv preprint arXiv:2509.15216 (2025)

    work page arXiv 2025

  33. [33]

    Bokai Chen, Weiwei Zheng, Liang Zhao, and Xiaojun Ding. 2025. Leveraging large language models to assist philosophical counseling: prospective techniques, value, and challenges.Humanities and Social Sciences Communications(2025). doi:10.1057/s41599-025-04657-7

    work page doi:10.1057/s41599-025-04657-7 2025

  34. [34]

    Chuke Chen, Nan Li, Jianchuan Qi, Huimin Chang, Wenjie Shi, Jinliang Xie, Jiayi Yuan, Hang Yang, Jing Guo, Changqing Xu, and Ming Xu

  35. [35]

    Leveraging LLMs for Environmental Complexity: Structured Fine-Tuning Data Sets and Deployment Strategies.Environmental Science & Technology60, 1 (2025)

    2025

  36. [36]

    Cheng-Yeh Chen and Raghupathy Sivakumar. 2026. The Mind’s Transformer: Computational Neuroanatomy of LLM-Brain Alignment. InThe Fourteenth International Conference on Learning Representations. https://openreview.net/forum?id=PgIlCCNxdB

    2026

  37. [37]

    Danlu Chen et al. 2024. LogogramNLP: Comparing Visual and Textual Representations of Ancient Logographic Writing Systems for NLP. In Findings of the Association for Computational Linguistics: EMNLP 2024. https://arxiv.org/abs/2408.04628

    work page arXiv 2024

  38. [38]

    Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, and Benyou Wang. 2025. Towards Medical Complex Reasoning with LLMs through Medical Verifiable Problems. InFindings of the Association for Computational Linguistics: ACL 2025 / preprint arXiv:2412.18925, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.18653/v1/2025.findings-acl.751 2025

  39. [39]

    Liuqing Chen, Qianzhi Jing, Yixin Tsang, Qianyi Wang, Lingyun Sun, and Jianxi Luo. 2024. DesignFusion: Integrating Generative Models for Conceptual Design Enrichment.Journal of Mechanical Design146, 11 (05 2024), 111703. doi:10.1115/1.4065487

    work page doi:10.1115/1.4065487 2024

  40. [40]

    Mouxiang Chen, Lei Zhang, Yunlong Feng, Xuwu Wang, Wenting Zhao, Ruisheng Cao, Jiaxi Yang, Jiawei Chen, Mingze Li, Zeyao Ma, Hao Ge, Zongmeng Zhang, Zeyu Cui, Dayiheng Liu, Jingren Zhou, Jianling Sun, Junyang Lin, and Binyuan Hui. 2026. SWE-Universe: Scale Real-World Verifiable Environments to Millions. arXiv:2602.02361 [cs.SE] https://arxiv.org/abs/2602.02361

    work page arXiv 2026

  41. [41]

    Wei Chen, Han Ding, Meng Yuan, Zhao Zhang, Deqing Wang, and Fuzhen Zhuang. 2025. Bridging Social Psychology and LLM Reasoning: Conflict-Aware Meta-Review Generation via Cognitive Alignment.arXiv preprint arXiv:2503.13879(2025). https://arxiv.org/pdf/2503.13879

    work page arXiv 2025

  42. [42]

    Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen. 2022. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks.arXiv preprint arXiv:2211.12588(2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  43. [43]

    Zhiyu Chen, Yujie Lu, and William Wang. 2023. Empowering Psychotherapy with Large Language Models: Cognitive Distortion Detection through Diagnosis of Thought Prompting. InFindings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 4295–4304....

    work page doi:10.18653/v1/2023.findings-emnlp.284 2023

  44. [44]

    Zhou Chen, Xiao Wang, Xinan Zhang, Ming Lin, Yuanhong Liao, Juanzi Li, and Yuqi Bai. 2025. GeoFactory: an LLM performance enhancement framework for geoscience factual and inferential tasks.Big Earth Data9, 4 (2025), 714–746. doi:10.1080/20964471.2025.2506291

    work page doi:10.1080/20964471.2025.2506291 2025

  45. [45]

    Zi-Yi Chen, Fan-Kai Xie, Meng Wan, Yang Yuan, Miao Liu, Zong-Guo Wang, Sheng Meng, and Yan-Gang Wang. 2023. MatChat: A large language model and application service platform for materials science.Chinese Physics B32, 11 (nov 2023), 118104. doi:10.1088/1674-1056/ad04cb

    work page doi:10.1088/1674-1056/ad04cb 2023

  46. [46]

    Yuheng Cheng, Huan Zhao, Xiyuan Zhou, Junhua Zhao, Yuji Cao, Chao Yang, and Xinlei Cai. 2025. A large language model for advanced power dispatch.Scientific Reports15, 1 (2025), 8925. Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branches 19

    2025

  47. [47]

    Byounggook Cho, Gi-Young Lee, Junghyun Jung, Junyeop Kim, GunHo Park, Patrick C. N. Martin, Hyobin Kim, Jeein Oh, Jong-Soo Kim, Jongpil Kim, Tae-Hyung Kim, and Kyoung-Jae Won. 2026. PersonaAI: An Interactive Agentic-AI Framework for Autonomous Hypothesis Generation and Validation in Aging.bioRxiv(2026). doi:10.64898/2026.01.16.699755

    work page doi:10.64898/2026.01.16.699755 2026

  48. [48]

    As Eastern Powers, I Will Veto

    Jonghyeon Choi, Yeonjun Choi, Hyun-chul Kim, and Beakcheol Jang. 2026. “As Eastern Powers, I Will Veto. ”: An Investigation of Nation-Level Bias of Large Language Models in International Relations. InProceedings of the AAAI Conference on Artificial Intelligence 2026, Vol. 40. 30404–30412

    2026

  49. [49]

    Kamal Choudhary. 2025. DiffractGPT: Atomic Structure Determination from X-ray Diffraction Patterns Using a Generative Pretrained Transformer. Journal of Physical Chemistry Letters(2025). doi:10.1021/acs.jpclett.4c03137

    work page doi:10.1021/acs.jpclett.4c03137 2025

  50. [50]

    Yu, and Qingsong Wen

    Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jingheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S. Yu, and Qingsong Wen. 2025. LLM Agents for Education: Advances and Applications. InFindings of the Association for Computational Linguistics: EMNLP 2025, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.)...

    work page doi:10.18653/v1/2025.findings-emnlp.743 2025

  51. [51]

    Daniel J H Chung, Zhiqi Gao, Yurii Kvasiuk, Tianyi Li, Moritz Münchmeyer, Maja Rudolph, Frederic Sala, and Sai Chaitanya Tadepalli. 2025. Theoretical physics benchmark (TPBench)—a dataset and study of AI reasoning capabilities in theoretical physics.Machine Learning: Science and Technology6, 3 (sep 2025), 030505. doi:10.1088/2632-2153/adfcb0

    work page doi:10.1088/2632-2153/adfcb0 2025

  52. [52]

    Pierre Colombo, Telmo Pires, Malik Boudiaf, Rui Melo, Gabriel Hautreux, Etienne Malaboeuf, Johanne Charpentier, Dominic Culver, and Michael Desa. 2024. SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domain. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems. https://openreview.net/forum?id=NLUYZ4ZqNq

    2024

  53. [53]

    Sergio Cruzes. 2025. Revolutionizing optical networks: The integration and impact of large language models.Optical Switching and Networking57 (2025), 100812

    2025

  54. [54]

    Hao Cui, Zahra Shamsi, Gowoon Cheon, et al. 2025. CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning. arXiv:2503.13517 [cs.CL] https://arxiv.org/abs/2503.13517

    work page arXiv 2025

  55. [55]

    Yiming Cui, X Yao, Y Qin, et al. 2025. Evaluating large language models on multimodal chemistry olympiad exams.Communications Chemistry (2025). doi:10.1038/s42004-025-01782-x

    work page doi:10.1038/s42004-025-01782-x 2025

  56. [56]

    Chongyuan Dai, Jinpeng Hu, Hongchang Shi, Zhuo Li, Xun Yang, and Meng Wang. 2025. Psyche-R1: Towards Reliable Psychological LLMs through Unified Empathy, Expertise, and Reasoning. arXiv:2508.10848 [cs.CL] https://arxiv.org/abs/2508.10848

    work page arXiv 2025

  57. [57]

    Xi Dai, Yu Xi, Yong Hu, Qingyan Ding, Yu Zhang, Hui Liu, Piaofei Chen, Xi Wang, Wang Wenjun, and Chaoxue Zhang. 2025. LLM Evaluation for Thyroid Nodule Assessment: Comparing ACR-TIRADS, C-TIRADS, and Clinician-AI Trust Gap.Frontiers in Endocrinology16 (2025), 1667809. doi:10.3389/fendo.2025.1667809

    work page doi:10.3389/fendo.2025.1667809 2025

  58. [58]

    Xin Dai, Buqiang Xu, Zhenghao Liu, Yukun Yan, Huiyuan Xie, Xiaoyuan Yi, Shuo Wang, and Ge Yu. 2026. LegalΔ: Enhancing Legal Reasoning in LLMS via Reinforcement Learning with Chain-Of-Thought Guided Information Gain. InICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 16912–16916. doi:10.1109/ICASSP55912....

    work page doi:10.1109/icassp55912.2026.11464256 2026

  59. [59]

    Anurag Das, David Kerr, Namino Glantz, Wendy Bevier, Rony Santiago, Ricardo Gutierrez-Osuna, and Bobak Mortazavi. 2025. CGMacros: a pilot scientific dataset for personalized nutrition and diet monitoring.Scientific Data12 (09 2025). doi:10.1038/s41597-025-05851-7

    work page doi:10.1038/s41597-025-05851-7 2025

  60. [60]

    Mattia De Cao et al. 2024. Deep Learning Meets Egyptology: a Hieroglyphic Transformer for Translating Ancient Egyptian. InProceedings of the 1st Workshop on Machine Learning for Ancient Languages. Association for Computational Linguistics. https://aclanthology.org/2024.ml4al-1.9v1.pdf

    2024

  61. [61]

    Tijmen de Haan, Yuan-Sen Ting, Tirthankar Ghosal, Tuan Dung Nguyen, Alberto Accomazzi, Emily Herron, Vanessa Lama, Rui Pan, Azton Wells, and Nesar Ramachandra. 2025. AstroMLab 4: Benchmark-Topping Performance in Astronomy Q&A with a 70B-Parameter Domain-Specialized Reasoning Model. arXiv:2505.17592 [astro-ph.IM] https://arxiv.org/abs/2505.17592

    work page arXiv 2025

  62. [62]

    Andrea Gregor de Varda, Ferdinando Pio D’Elia, Hope Kean, Andrew Lampinen, and Evelina Fedorenko. 2025. The cost of thinking is similar between large reasoning models and humans.Proceedings of the National Academy of Sciences122, 47 (2025), e2520077122

    2025

  63. [63]

    DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, et al. 2025. DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models. arXiv:2512.02556 [cs.CL] https://arxiv.org/abs/2512.02556

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  64. [64]

    Cheng Deng, Tianhang Zhang, Zhongmou He, Yi Xu, Qiyuan Chen, Yuanyuan Shi, Luoyi Fu, Weinan Zhang, Xinbing Wang, Chenghu Zhou, Zhouhan Lin, and Junxian He. 2023. K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization. arXiv:2306.05064 doi:10.48550/arXiv.2306.05064 Project page notes acceptance to WSDM 2024 (ACM DOI may be 1...

    work page doi:10.48550/arxiv.2306.05064 2023

  65. [65]

    Gebauer, Henry Alexander Bradley, Dawid Maciorowski, Bria Persaud, Jordan Despanie, et al

    Sunishchal Dev, Charles Teague, Grant Ellison, Kyle Brady, Jeffrey Lee, Sarah L. Gebauer, Henry Alexander Bradley, Dawid Maciorowski, Bria Persaud, Jordan Despanie, et al. 2025.Toward Comprehensive Benchmarking of the Biological Knowledge of Frontier Large Language Models. Research Report RRA3797-1. RAND Corporation

    2025

  66. [66]

    González-Castaño

    Francisco de Arriba Pérez, Silvia García-Méndez, Javier Otero-Mosquera, and Francisco J. González-Castaño. 2024. Explainable cognitive decline detection in free dialogues with a Machine Learning approach based on pre-trained Large Language Models.Applied Intelligence54, 24 (Sept. 2024), 12613–12628. doi:10.1007/s10489-024-05808-0

    work page doi:10.1007/s10489-024-05808-0 2024

  67. [67]

    Mehak Preet Dhaliwal, Andong Hua, Laya Pullela, Ryan Burke, and Yao Qin. 2025. NutriBench: A Dataset for Evaluating Large Language Models in Nutrition Estimation from Meal Descriptions. InThe Thirteenth International Conference on Learning Representations. https://openreview.net/ forum?id=6LtdZCyuZR

    2025

  68. [68]

    Zhehao Dong, Zhen Lu, and Yue Yang. 2025. Fine-tuning a large language model for automating computational fluid dynamics simulations. Theoretical and Applied Mechanics Letters15, 3 (may 2025), 100594. doi:10.1016/j.taml.2025.100594 20 Ferdinan et al

    work page doi:10.1016/j.taml.2025.100594 2025

  69. [69]

    Filip Dorm, Joseph Millard, Drew Purves, Michael Harfoot, and Oisin Mac Aodha. 2025. Large language models possess some ecological knowledge, but how much?bioRxiv(2025). doi:10.1101/2025.02.10.637097

    work page doi:10.1101/2025.02.10.637097 2025

  70. [70]

    Gardner, and Hao ‘Frank’ Yang

    Hongru Du, Yang Zhao, Jianan Zhao, Shaochong Xu, Xihong Lin, Yiran Chen, Lauren M. Gardner, and Hao ‘Frank’ Yang. 2025. Advancing real-time infectious disease forecasting using large language models.Nature Computational Science5, 6 (01 Jun 2025), 467–480. doi:10.1038/s43588-025-00798-6

    work page doi:10.1038/s43588-025-00798-6 2025

  71. [71]

    Mengge Du, Yuntian Chen, Zhongzheng Wang, Longfeng Nie, and Dongxiao Zhang. 2024. Large language models for automatic equation discovery of nonlinear dynamics.Physics of Fluids36, 9 (09 2024), 097121. doi:10.1063/5.0224297

    work page doi:10.1063/5.0224297 2024

  72. [72]

    Plasek, Ya-Wen Chuang, Liqin Wang, Gad A

    Xinsong Du, John Novoa-Laurentiev, Joseph M. Plasek, Ya-Wen Chuang, Liqin Wang, Gad A. Marshall, Stephanie K. Mueller, Frank Chang, Surabhi Datta, Hunki Paek, Bin Lin, Qiang Wei, Xiaoyan Wang, Jingqi Wang, Hao Ding, Frank J. Manion, Jingcheng Du, David W. Bates, and Li Zhou. 2024. Enhancing early detection of cognitive decline in the elderly: a comparativ...

    work page doi:10.1016/j.ebiom.2024.105401 2024

  73. [73]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2025. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv:2404.16130 [cs.CL] https://arxiv.org/abs/2404.16130

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  74. [74]

    Ehrlich-Sommer, B

    F. Ehrlich-Sommer, B. Eberhard, and A. Holzinger. 2025. ForestGPT and Beyond: A Trustworthy Domain-Specific Large Language Model Paving the Way to Forestry 5.0.Electronics14, 18 (2025), 3583. https://www.mdpi.com/2079-9292/14/18/3583

    2025

  75. [75]

    Morris, Chun-Cheng Chang, Xuhai "Orson" Xu, Lianhui Qin, Daniel McDuff, Xin Liu, Shwetak Patel, and Vikram Iyer

    Zachary Englhardt, Chengqian Ma, Margaret E. Morris, Chun-Cheng Chang, Xuhai "Orson" Xu, Lianhui Qin, Daniel McDuff, Xin Liu, Shwetak Patel, and Vikram Iyer. 2024. From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models.Proc. ACM Interact. Mob. Wearable Ubiquitous Technol...

    work page doi:10.1145/3659604 2024

  76. [76]

    European Research Council. 2025. Panel Chairs – ERC Starting Grant 2026. https://erc.europa.eu/system/files/2025-10/Panel_Chairs_ERC_ Starting_Grant_2026.pdf. Accessed: 2026-02-11

    2025

  77. [77]

    Maddison, and BO WANG

    Adibvafa Fallahpour, Andrew Magnuson, Purav Gupta, Shihao Ma, Jack Naimer, Arnav Shah, Haonan Duan, Omar Ibrahim, Hani Goodarzi, Chris J. Maddison, and BO WANG. 2025. BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems. https://openreview.net/forum?i...

    2025

  78. [78]

    Ka Siu Fan, Jeffrey Gan, Isabelle Zou, Maja Kaladjiska, Monique Borg Inguanez, and Gillian Garden. 2025. Poor Performance of Large Language Models Based on the Diabetes and Endocrinology Specialty Certificate Examination of the United Kingdom.Cureus17 (10 2025). doi:10.7759/ cureus.93960

    2025

  79. [79]

    Yu Fan, Jingwei Ni, Jakob Merane, Yang Tian, Yoan Hermstrüwer, Yinya Huang, Mubashara Akhtar, Etienne Salimbeni, Florian Geering, Oliver Dreyer, Daniel Brunner, Markus Leippold, Mrinmaya Sachan, Alexander Stremitzer, Christoph Engel, Elliott Ash, and Joel Niklaus. 2026. LEXam: Benchmarking Legal Reasoning on 340 Law Exams. InThe Fourteenth International C...

    2026

  80. [80]

    Yin Fang, Qiao Jin, Guangzhi Xiong, Bowen Jin, Xianrui Zhong, Siru Ouyang, Aidong Zhang, Jiawei Han, and Zhiyong Lu. 2025. Cell-o1: Training LLMs to Solve Single-Cell Reasoning Puzzles with Reinforcement Learning. arXiv:2506.02911 [cs.CL] https://arxiv.org/abs/2506.02911

    work page arXiv 2025

Showing first 80 references.