Towards A Generative Protein Evolution Machine with DPLM-Evo

Jiasheng Ye; Liang Hong; Quanquan Gu; Shujian Huang; Xinyou Wang; Yu Li; Zaixiang Zheng

DPLM-Evo models protein evolution as accumulated edits rather than masks to improve mutation prediction and enable flexible generation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-07-01 08:03 UTC pith:BF7337SO

load-bearing objection DPLM-Evo adds explicit indel and substitution modeling to discrete diffusion via an upsampled latent alignment and context-aware noising, which is a direct response to the masking mismatch in prior work.

arxiv 2605.00182 v3 pith:BF7337SO submitted 2026-04-30 cs.LG

Towards A Generative Protein Evolution Machine with DPLM-Evo

Xinyou Wang , Liang Hong , Jiasheng Ye , Zaixiang Zheng , Yu Li , Shujian Huang , Quanquan Gu This is my paper

classification cs.LG

keywords protein language modelsdiscrete diffusionevolutionary modelingmutation effect predictionindel operationsprotein designgenerative modelssequence editing

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to demonstrate that standard masked diffusion in protein language models misses the biological process of gradual substitution, insertion, and deletion. It introduces an alternative diffusion process that decouples a fixed upsampled latent alignment space from the actual variable-length sequences and applies a context-dependent noising kernel to generate realistic mutation patterns. If this holds, the result is stronger performance on tasks that require understanding how proteins change, plus the ability to simulate evolutionary trajectories of any length and to edit existing proteins along explicit paths. A reader would care because such a shift could make computational protein design and optimization more aligned with how real sequences arise under selection pressure.

Core claim

DPLM-Evo is presented as an evolutionary discrete diffusion framework that explicitly predicts substitution, insertion, and deletion operations during denoising. It achieves this by decoupling an upsampled-length latent alignment space from the variable-length observed sequence space and by introducing a contextualized evolutionary noising kernel that produces biologically informed, context-dependent mutation patterns. On this basis the method reports state-of-the-art mutation effect prediction on ProteinGym in the single-sequence setting and demonstrates variable-length simulated evolution together with post-editing of existing proteins via explicit edit trajectories.

What carries the argument

The decoupled upsampled-length latent alignment space combined with the contextualized evolutionary noising kernel that lets the model predict substitution, insertion, and deletion operations explicitly.

Load-bearing premise

That separating a latent alignment space from observed sequences and using context-dependent noise will produce mutation patterns that match real evolutionary constraints better than masked diffusion and without introducing artifacts in the generated sequences.

What would settle it

A head-to-head evaluation on ProteinGym in which DPLM-Evo does not exceed prior single-sequence methods on mutation effect prediction, or inspection of generated sequences that reveals systematic non-functional artifacts absent from natural proteins.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Sequence understanding improves because the model directly encodes substitution, insertion, and deletion operations.
Mutation effect prediction reaches state-of-the-art results on ProteinGym under the single-sequence setting.
Variable-length simulated evolution becomes possible by traversing explicit edit trajectories.
Existing proteins can be post-edited and optimized by following the same explicit edit paths.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The explicit edit trajectories could make it easier to incorporate additional constraints such as structural stability during generation.
If the approach scales, it might support iterative design loops in which a protein is evolved in simulation toward a desired function before laboratory testing.
The separation of latent alignment from observed length may generalize to other variable-length sequence domains where insertion and deletion matter.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

DPLM-Evo adds explicit indel and substitution modeling to discrete diffusion via an upsampled latent alignment and context-aware noising, which is a direct response to the masking mismatch in prior work.

read the letter

The core move is shifting from masked diffusion to a process that predicts substitutions, insertions, and deletions as the denoising steps. This matches the biological story of accumulated edits more closely than unmasking does.

The new pieces are the upsampled-length latent alignment that decouples the hidden space from observed sequence length, plus the contextualized evolutionary noising kernel that conditions mutation rates on surrounding residues. These let the model generate variable-length sequences and produce explicit edit trajectories for post-editing existing proteins. The abstract positions this as enabling simulated evolution and better single-sequence mutation effect prediction on ProteinGym.

If the experiments show clear gains over standard DPLM baselines with the same data and compute, the framing could be useful for people who need controllable length and edit-based optimization. The approach is straightforward to understand and builds on existing discrete diffusion without adding heavy new machinery.

The main soft spot is that the abstract supplies no equations, no quantitative results, and no ablation details, so the SOTA claim and the claim that the kernel produces biologically informed patterns cannot be checked. The decoupling of latent and observed spaces could introduce its own length artifacts or training instabilities, but nothing in the provided text lets us evaluate that risk. No code or data release is mentioned.

This paper is for groups already working on diffusion or autoregressive protein models who want an edit-centric alternative. A reader focused on ProteinGym benchmarks or directed evolution applications would get the most from it.

It deserves a serious referee because the conceptual shift is distinct and the tasks are relevant, even if the current evidence is limited to the abstract. Send it for review once the full methods and results are in hand.

Referee Report

0 major / 2 minor

Summary. The paper introduces DPLM-Evo, a discrete diffusion protein language model that explicitly models evolutionary edit operations (substitutions, insertions, deletions) rather than relying on masked diffusion. It decouples an upsampled latent alignment space from variable-length observed sequences to enable indel-aware generation, and introduces a contextualized evolutionary noising kernel for biologically informed mutation patterns. Claims include SOTA mutation effect prediction on ProteinGym in the single-sequence setting, plus capabilities for variable-length simulated evolution and explicit edit-trajectory post-editing/optimization of proteins.

Significance. If the empirical claims hold, the work would provide a more biologically aligned generative framework for proteins than standard masked DPLMs, potentially improving both predictive accuracy on mutation effects and controllable generation/editing tasks. The explicit modeling of edit operations and the latent alignment decoupling address a clear mismatch between current diffusion objectives and evolutionary processes.

minor comments (2)

The abstract references ProteinGym results and SOTA performance but provides no details on baselines, metrics, or single-sequence setting definition; these should be expanded in the main text with explicit comparisons.
Notation for the latent alignment space and contextualized noising kernel is introduced without equations in the provided abstract; full definitions and any associated loss terms should be presented clearly in §3 or §4.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their summary of DPLM-Evo and for noting its potential alignment with evolutionary processes. The report provides no specific major comments, so we have no point-by-point responses to offer at this stage. We remain available to address any additional questions or concerns.

Circularity Check

0 steps flagged

No significant circularity; abstract-only text contains no equations or self-referential derivations

full rationale

The provided abstract describes DPLM-Evo conceptually (decoupling latent alignment space, contextualized evolutionary noising kernel, explicit edit trajectories) but contains zero equations, parameter-fitting procedures, or citations. No load-bearing step reduces to a self-definition, fitted input renamed as prediction, or self-citation chain. This matches the default expectation of a non-circular paper when no derivation chain is exhibited; the reader's 5.0 score reflects absence of assessable content rather than detected circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no information on free parameters, axioms, or invented entities; ledger left empty.

pith-pipeline@v0.9.1-grok · 5774 in / 1061 out tokens · 21918 ms · 2026-07-01T08:03:17.445605+00:00 · methodology

0 comments

read the original abstract

Proteins are shaped by gradual evolution under biophysical and functional constraints. Protein language models learn rich evolutionary constraints from large-scale sequences, and discrete diffusion-based protein language models~(\eg, DPLMs) are promising for both understanding and generation. However, existing DPLMs typically rely on masked diffusion that contradicts a simple biological intuition: proteins evolve through accumulated edits, not by emerging from masks. Consequently, these frameworks lack explicit pretraining objectives for substitution and insertion/deletion (indel) operations, limiting both optimization-style post-editing and flexible guided generation. To address these limitations, we present DPLM-Evo, an evolutionary discrete diffusion framework that explicitly predicts substitution, insertion, and deletion operations during denoising. DPLM-Evo decouples an upsampled-length latent alignment space from the variable-length observed sequence space, which makes indel-aware generation tractable. To better align substitutions with real evolution, we further introduce a contextualized evolutionary noising kernel that produces biologically informed, context-dependent mutation patterns. Across tasks, DPLM-Evo improves sequence understanding and achieves state-of-the-art mutation effect prediction performance on ProteinGym in the single-sequence setting. It also enables variable-length simulated evolution, and post-editing/optimization of existing proteins via explicit edit trajectories.

Review history (3 revisions) →

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

AMix-2: Establishing Protein as a Native Modality in Large Language Models
q-bio.BM 2026-05 unverdicted novelty 6.0

AMix-2 unifies protein sequences and text in one LLM via shared tokens and block-wise diffusion modeling, introduces the ProteinArena benchmark, and reports competitive performance against task-specific protein models...

Reference graph

Works this paper leans on

61 extracted references · 16 canonical work pages · cited by 1 Pith paper · 2 internal anchors

[1]

Protein generation with evolutionary diffusion: sequence is all you need.bioRxiv, pages 2023–09, 2023

Sarah Alamdari, Nitya Thakkar, Rianne van den Berg, Alex Xijie Lu, Nicolo Fusi, Ava Pardis Amini, and Kevin K Yang. Protein generation with evolutionary diffusion: sequence is all you need.bioRxiv, pages 2023–09, 2023

2023
[2]

Structured denoising diffusion models in discrete state-spaces

Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. In Advances in Neural Information Processing Systems, volume 34, pages 17981–17993, 2021

2021
[3]

A diffusion model to shrink proteins while maintaining their function.arXiv preprint arXiv:2511.07390, 2025

Ethan Baron, Alan N Amin, Ruben Weitzman, Debora Marks, and Andrew Gordon Wilson. A diffusion model to shrink proteins while maintaining their function.arXiv preprint arXiv:2511.07390, 2025

work page arXiv 2025
[4]

Protein sequence profile prediction using protalbert transformer.Computational Biology and Chemistry, 99:107717, 2022

Armin Behjati, Fatemeh Zare-Mirakabad, Seyed Shahriar Arab, and Abbas Nowzari-Dalini. Protein sequence profile prediction using protalbert transformer.Computational Biology and Chemistry, 99:107717, 2022

2022
[5]

Proteinbert: a universal deep- learning model of protein sequence and function.Bioinformatics, 38(8):2102–2110, 2022

Nadav Brandes, Dan Ofer, Yam Peleg, Nadav Rappoport, and Michal Linial. Proteinbert: a universal deep- learning model of protein sequence and function.Bioinformatics, 38(8):2102–2110, 2022

2022
[6]

Famsa: Fast and accurate multiple se- quence alignment of huge protein families.Scientific reports, 6(1):33964, 2016

Sebastian Deorowicz, Agnieszka Debudaj-Grabysz, and Adam Gudyś. Famsa: Fast and accurate multiple se- quence alignment of huge protein families.Scientific reports, 6(1):33964, 2016

2016
[7]

Prottrans: Toward understanding the language of life through self-supervised learning

Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. Prottrans: Toward understanding the language of life through self-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 44(10): 7112–7127, 2021

2021
[8]

Esm cambrian: Revealing the mysteries of proteins with unsupervised learning, 2024

ESM Team. Esm cambrian: Revealing the mysteries of proteins with unsupervised learning, 2024. URLhttps: //evolutionaryscale.ai/blog/esm-cambrian

2024
[9]

Disease variant prediction with deep generative models of evolutionary data.Nature, 599(7883):91–95, 2021

JonathanFrazer, PascalNotin, MafaldaDias, AidanGomez, JosephKMin, KellyBuss, DanielHZuber, JosephN Glover, and Debora S Marks. Disease variant prediction with deep generative models of evolutionary data.Nature, 599(7883):91–95, 2021

2021
[10]

Scaling diffusion language models via adaptation from autoregressive models

Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, Hao Peng, and Lingpeng Kong. Scaling diffusion language models via adaptation from autoregressive models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=j1tSLYKwg8

2025
[11]

Connectionist temporal clas- sification: labelling unsegmented sequence data with recurrent neural networks

Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal clas- sification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pages 369–376, 2006

2006
[12]

Protein design with guided discrete diffusion

Nate Gruver, Samuel Stanton, Nathan C Frey, Tim GJ Rudner, Isidro Hotber, Julien Lafrance-Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew Gordon Wilson. Protein design with guided discrete diffusion. InAdvances in Neural Information Processing Systems, 2023

2023
[13]

Fully non-autoregressive neural machine translation: Tricks of the trade

Jiatao Gu and Xiang Kong. Fully non-autoregressive neural machine translation: Tricks of the trade. InFindings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 120–133, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.findings-acl.11. URLhttps://aclanthology. org/2021.findings-acl.11

work page doi:10.18653/v1/2021.findings-acl.11 2021
[14]

Levenshtein transformer

Jiatao Gu, Changhan Wang, and Junbo Zhao. Levenshtein transformer. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors,Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver,BC, Ca...

2019
[15]

Nat: Neural architecture transformer for accurate and compact architectures.Advances in Neural Information Processing Systems, 32, 2019

Yong Guo, Yin Zheng, Mingkui Tan, Qi Chen, Jian Chen, Peilin Zhao, and Junzhou Huang. Nat: Neural architecture transformer for accurate and compact architectures.Advances in Neural Information Processing Systems, 32, 2019

2019
[16]

arXiv preprint arXiv:2506.09018 , year=

Marton Havasi, Brian Karrer, Itai Gat, and Ricky TQ Chen. Edit flows: Flow matching with edit operations. arXiv preprint arXiv:2506.09018, 2025

work page arXiv 2025
[17]

Simulating 500 million years of evolution with a language model

TomasHayes, RoshanRao, HalilAkin, NicholasJSofroniew, DenizOktay, ZemingLin, RobertVerkuil, VincentQ Tran, Jonathan Deaton, Marius Wiggert, et al. Simulating 500 million years of evolution with a language model. bioRxiv, pages 2024–07, 2024

2024
[18]

Diffusionbert: Improving gen- erative masked language models with diffusion models

Zhengfu He, Tianxiang Sun, Kuanning Wang, Xuanjing Huang, and Xipeng Qiu. Diffusionbert: Improving gen- erative masked language models with diffusion models. InAnnual Meeting of the Association for Computational Linguistics, 2023

2023
[19]

Denoising diffusion probabilistic models

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020. URLhttps://proceedings.neurips.cc/paper/2020/ file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf

2020
[20]

Argmax flows and multinomial diffusion: Learning categorical distributions

Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34:12454– 12465, 2021

2021
[21]

Elucidatingthedesignspaceofmultimodalproteinlanguagemodels

Cheng-Yen Hsieh, Xinyou Wang, Daiheng Zhang, Dongyu Xue, Fei Ye, Shujian Huang, Zaixiang Zheng, and QuanquanGu. Elucidatingthedesignspaceofmultimodalproteinlanguagemodels. In Forty-secondInternational Conference on Machine Learning, 2025

2025
[22]

Learning inverse folding from millions of predicted structures

Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer, and Alexander Rives. Learning inverse folding from millions of predicted structures. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors,Proceedings of the 39th International Conference on Machine Learning, volume 162 ofP...

2022
[23]

Gemme: a simple and fast global epistatic model predicting mutational effects.Molecular biology and evolution, 36(11):2604–2619, 2019

Elodie Laine, Yasaman Karami, and Alessandra Carbone. Gemme: a simple and fast global epistatic model predicting mutational effects.Molecular biology and evolution, 36(11):2604–2619, 2019

2019
[24]

Language models of protein sequences at the scale of evolution enable accurate structure prediction.BioRxiv, 2022

Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, et al. Language models of protein sequences at the scale of evolution enable accurate structure prediction.BioRxiv, 2022

2022
[25]

Evolutionary-scale prediction of atomic-level protein structure with a language model

Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637):1123–1130, 2023

2023
[26]

arXiv preprint arXiv:2509.24007 , year=

Yangzhou Liu, Yue Cao, Hao Li, Gen Luo, Zhe Chen, Weiyun Wang, Xiaobo Liang, Biqing Qi, Lijun Wu, Changyao Tian, et al. Sequential diffusion language models.arXiv preprint arXiv:2509.24007, 2025

work page arXiv 2025
[27]

Expert-guided pro- tein language models enable accurate and blazingly fast fitness prediction.Bioinformatics, 40(11):btae621, 11

Céline Marquet, Julius Schlensok, Marina Abakarova, Burkhard Rost, and Elodie Laine. Expert-guided pro- tein language models enable accurate and blazingly fast fitness prediction.Bioinformatics, 40(11):btae621, 11
[28]

doi: 10.1093/bioinformatics/btae621

ISSN 1367-4811. doi: 10.1093/bioinformatics/btae621. URLhttps://doi.org/10.1093/bioinformatics/ btae621

work page doi:10.1093/bioinformatics/btae621
[29]

Language models enable zero-shot prediction of the effects of mutations on protein function

Joshua Meier, Roshan Rao, Robert Verkuil, Jason Liu, Tom Sercu, and Alex Rives. Language models enable zero-shot prediction of the effects of mutations on protein function. InAdvancesin Neural Information Processing Systems, pages 29287–29303, 2021

2021
[30]

Dima: Diffusion mamba – a diffusion model with state space backbone for protein design

Alexey Meshchaninov, Daniil Zinchenko, Andrey Golovin, Sergey Evfratov, Alexey Chertkov, and Nikita Nikitin. Dima: Diffusion mamba – a diffusion model with state space backbone for protein design. arXiv preprint arXiv:2410.13514, 2024

work page arXiv 2024
[31]

Transforming the language of life: transformer neural networks for protein prediction tasks

Ananthan Nambiar, Maeve Heflin, Simon Liu, Sergei Maslov, Mark Hopkins, and Anna Ritz. Transforming the language of life: transformer neural networks for protein prediction tasks. In Proceedings of the 11th ACM international conference on bioinformatics, computational biology and health informatics, pages 1–8, 2020. 14

2020
[32]

Scaling up masked diffusion models on text

Shen Nie, Fengqi Zhu, Chao Du, Tianyu Pang, Qian Liu, Guangtao Zeng, Min Lin, and Chongxuan Li. Scaling up masked diffusion models on text. InThe Thirteenth International Conference on Learning Representations, 2024

2024
[33]

Large Language Diffusion Models

Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models.arXiv preprint arXiv:2502.09992, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025
[34]

Progen2: exploring the boundaries of protein language models.arXiv preprint arXiv:2206.13517, 2022

Erik Nijkamp, Jeffrey Ruffolo, Eli N Weinstein, Nikhil Naik, and Ali Madani. Progen2: exploring the boundaries of protein language models.arXiv preprint arXiv:2206.13517, 2022

work page arXiv 2022
[35]

Proteingym: Large-scale benchmarks for protein fitness prediction and design

Pascal Notin, Aaron W Kollasch, Daniel Ritter, Lood Van Niekerk, Steffan Paul, Han Spinner, Nathan J Rollins, Ada Shaw, Ruben Weitzman, Jonathan Frazer, et al. Proteingym: Large-scale benchmarks for protein fitness prediction and design. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023

2023
[36]

Your absorbing discrete diffusion secretly models the conditional distributions of clean data

Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. InThe Thirteenth International Conference on Learning Representations, 2024

2024
[37]

Evaluating protein transfer learning with tape

Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Peter Chen, John Canny, Pieter Abbeel, and Yun Song. Evaluating protein transfer learning with tape. Advances in neural information processing systems, 32, 2019

2019
[38]

Diffuser: Diffusion via edit-based reconstruction

Machel Reid, Vincent Josua Hellendoorn, and Graham Neubig. Diffuser: Diffusion via edit-based reconstruction. In International Conference on Learning Representations, 2022

2022
[39]

Lawrence Zitnick, Jerry Ma, and Rob Fergus

Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C. Lawrence Zitnick, Jerry Ma, and Rob Fergus. Biological structure and function emerge from scal- ing unsupervised learning to 250 million protein sequences. PNAS, 2019. doi: 10.1101/622803. URL https://www.biorxiv.org/content/10.1101/622803v4

work page doi:10.1101/622803 2019
[40]

Simpleandeffectivemaskeddiffusionlanguagemodels

Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, andVolodymyrKuleshov. Simpleandeffectivemaskeddiffusionlanguagemodels. AdvancesinNeuralInformation Processing Systems, 37:130136–130184, 2024

2024
[41]

The diffusion duality

Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin T Chiu, and Volodymyr Kuleshov. The diffusion duality. InForty-secondInternational Conference on Machine Learning, 2025

2025
[42]

Simple guidance mechanisms for discrete diffusion models

Yair Schiff, Subham Sekhar Sahoo, Hao Phung, Guanghan Wang, Sam Boshar, Hugo Dalla-torre, Bernardo P de Almeida, Alexander M Rush, Thomas PIERROT, and Volodymyr Kuleshov. Simple guidance mechanisms for discrete diffusion models. InThe Thirteenth International Conference on Learning Representations, 2025

2025
[43]

Simplified and generalized masked diffusion for discrete data.Advancesin neural information processing systems, 37:103131–103167, 2024

Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and generalized masked diffusion for discrete data.Advancesin neural information processing systems, 37:103131–103167, 2024

2024
[44]

Deep unsupervised learning using nonequilibrium thermodynamics

Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Francis Bach and David Blei, editors,International Conference on Machine Learning, volume 37 ofProceedings of Machine Learning Research, pages 2256–2265, Lille, France, 07–09 Jul
[45]

URLhttps://proceedings.mlr.press/v37/sohl-dickstein15.html

PMLR, PMLR. URLhttps://proceedings.mlr.press/v37/sohl-dickstein15.html
[46]

Denoising diffusion implicit models

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2020

2020
[47]

Score- based generative modeling through stochastic differential equations

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score- based generative modeling through stochastic differential equations. InInternational Conference on Learning Representations, 2020

2020
[48]

Saprot: Protein language modeling with structure-aware vocabulary.bioRxiv, pages 2023–10, 2023

Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. Saprot: Protein language modeling with structure-aware vocabulary.bioRxiv, pages 2023–10, 2023

2023
[49]

PoET: A generative model of protein families as sequences-of-sequences

Timothy F Truong Jr and Tristan Bepler. PoET: A generative model of protein families as sequences-of-sequences. In Advancesin Neural Information Processing Systems, 2024

2024
[50]

Language models generalize beyond natural proteins.bioRxiv, pages 2022–12, 2022

Robert Verkuil, Ori Kabeli, Yilun Du, Basile IM Wicky, Lukas F Milles, Justas Dauparas, David Baker, Sergey Ovchinnikov, Tom Sercu, and Alexander Rives. Language models generalize beyond natural proteins.bioRxiv, pages 2022–12, 2022. 15

2022
[51]

Gen- eralized interpolating discrete diffusion

Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, and Thomas Hofmann. Gen- eralized interpolating discrete diffusion. InForty-secondInternational Conference on Machine Learning, 2025

2025
[52]

arXiv preprint arXiv:2410.13782 , year=

Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, and Quanquan Gu. Dplm-2: A multimodal diffusion protein language model.arXiv preprint arXiv:2410.13782, 2024

work page arXiv 2024
[53]

arXiv preprint arXiv:2402.18567 , year=

Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, and Quanquan Gu. Diffusion language models are versatile protein learners.arXiv preprint arXiv:2402.18567, 2024

work page arXiv 2024
[54]

Dreamon: Diffusion language models for code infilling beyond fixed-size canvas, 2025

Zirui Wu, Lin Zheng, Zhihui Xie, Jiacheng Ye, Jiahui Gao, Yansong Feng, Zhenguo Li, Victoria W., Guorui Zhou, and Lingpeng Kong. Dreamon: Diffusion language models for code infilling beyond fixed-size canvas, 2025. URL https://hkunlp.github.io/blog/2025/dreamon

2025
[55]

Modeling protein using large-scale pretrain language model

Yijia Xiao, Jiezhong Qiu, Ziang Li, Chang-Yu Hsieh, and Jie Tang. Modeling protein using large-scale pretrain language model. arXiv preprint arXiv:2108.07435, 2021

work page arXiv 2021
[56]

Convolutions are competitive with transformers for protein sequence pretraining

Kevin K Yang, Alex X Lu, and Nicolo Fusi. Convolutions are competitive with transformers for protein sequence pretraining. bioRxiv, pages 2022–05, 2022

2022
[57]

Dream 7B: Diffusion Large Language Models

Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b: Diffusion large language models.arXiv preprint arXiv:2508.15487, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025
[58]

arXiv preprint arXiv:2308.12219 , year =

Jiasheng Ye, Zaixiang Zheng, Yu Bao, Lihua Qian, and Quanquan Gu. Diffusion language models can perform many tasks with scaling and instruction-finetuning.arXiv preprint arXiv:2308.12219, 2023

work page arXiv 2023
[59]

arXiv preprint arXiv:2302.10025 , year=

Jiasheng Ye, Zaixiang Zheng, Yu Bao, Lihua Qian, and Mingxuan Wang. Dinoiser: Diffused conditional sequence learning by manipulating noises.arXiv preprint arXiv:2302.10025, 2023

work page arXiv 2023
[60]

A reparameterized discrete diffusion model for text generation.arXiv preprint arXiv:2302.05737, 2023

Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong. A reparameterized discrete diffusion model for text generation. arXiv preprint arXiv:2302.05737, 2023

work page arXiv 2023
[61]

Structure-informed language models are protein designers

Zaixiang Zheng, Yifan Deng, Dongyu Xue, Yi Zhou, Fei YE, and Quanquan Gu. Structure-informed language models are protein designers. InInternational Conference on Machine Learning, 2023. 16 Appendix A Training Details. A.1 Substitution Learning with Contextualized Evolutionary Noise The quality of the DPLM-Evo heavily depends on how the substitution proces...

2023

[1] [1]

Protein generation with evolutionary diffusion: sequence is all you need.bioRxiv, pages 2023–09, 2023

Sarah Alamdari, Nitya Thakkar, Rianne van den Berg, Alex Xijie Lu, Nicolo Fusi, Ava Pardis Amini, and Kevin K Yang. Protein generation with evolutionary diffusion: sequence is all you need.bioRxiv, pages 2023–09, 2023

2023

[2] [2]

Structured denoising diffusion models in discrete state-spaces

Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. In Advances in Neural Information Processing Systems, volume 34, pages 17981–17993, 2021

2021

[3] [3]

A diffusion model to shrink proteins while maintaining their function.arXiv preprint arXiv:2511.07390, 2025

Ethan Baron, Alan N Amin, Ruben Weitzman, Debora Marks, and Andrew Gordon Wilson. A diffusion model to shrink proteins while maintaining their function.arXiv preprint arXiv:2511.07390, 2025

work page arXiv 2025

[4] [4]

Protein sequence profile prediction using protalbert transformer.Computational Biology and Chemistry, 99:107717, 2022

Armin Behjati, Fatemeh Zare-Mirakabad, Seyed Shahriar Arab, and Abbas Nowzari-Dalini. Protein sequence profile prediction using protalbert transformer.Computational Biology and Chemistry, 99:107717, 2022

2022

[5] [5]

Proteinbert: a universal deep- learning model of protein sequence and function.Bioinformatics, 38(8):2102–2110, 2022

Nadav Brandes, Dan Ofer, Yam Peleg, Nadav Rappoport, and Michal Linial. Proteinbert: a universal deep- learning model of protein sequence and function.Bioinformatics, 38(8):2102–2110, 2022

2022

[6] [6]

Famsa: Fast and accurate multiple se- quence alignment of huge protein families.Scientific reports, 6(1):33964, 2016

Sebastian Deorowicz, Agnieszka Debudaj-Grabysz, and Adam Gudyś. Famsa: Fast and accurate multiple se- quence alignment of huge protein families.Scientific reports, 6(1):33964, 2016

2016

[7] [7]

Prottrans: Toward understanding the language of life through self-supervised learning

Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. Prottrans: Toward understanding the language of life through self-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 44(10): 7112–7127, 2021

2021

[8] [8]

Esm cambrian: Revealing the mysteries of proteins with unsupervised learning, 2024

ESM Team. Esm cambrian: Revealing the mysteries of proteins with unsupervised learning, 2024. URLhttps: //evolutionaryscale.ai/blog/esm-cambrian

2024

[9] [9]

Disease variant prediction with deep generative models of evolutionary data.Nature, 599(7883):91–95, 2021

JonathanFrazer, PascalNotin, MafaldaDias, AidanGomez, JosephKMin, KellyBuss, DanielHZuber, JosephN Glover, and Debora S Marks. Disease variant prediction with deep generative models of evolutionary data.Nature, 599(7883):91–95, 2021

2021

[10] [10]

Scaling diffusion language models via adaptation from autoregressive models

Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, Hao Peng, and Lingpeng Kong. Scaling diffusion language models via adaptation from autoregressive models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=j1tSLYKwg8

2025

[11] [11]

Connectionist temporal clas- sification: labelling unsegmented sequence data with recurrent neural networks

Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal clas- sification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pages 369–376, 2006

2006

[12] [12]

Protein design with guided discrete diffusion

Nate Gruver, Samuel Stanton, Nathan C Frey, Tim GJ Rudner, Isidro Hotber, Julien Lafrance-Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew Gordon Wilson. Protein design with guided discrete diffusion. InAdvances in Neural Information Processing Systems, 2023

2023

[13] [13]

Fully non-autoregressive neural machine translation: Tricks of the trade

Jiatao Gu and Xiang Kong. Fully non-autoregressive neural machine translation: Tricks of the trade. InFindings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 120–133, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.findings-acl.11. URLhttps://aclanthology. org/2021.findings-acl.11

work page doi:10.18653/v1/2021.findings-acl.11 2021

[14] [14]

Levenshtein transformer

Jiatao Gu, Changhan Wang, and Junbo Zhao. Levenshtein transformer. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors,Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver,BC, Ca...

2019

[15] [15]

Nat: Neural architecture transformer for accurate and compact architectures.Advances in Neural Information Processing Systems, 32, 2019

Yong Guo, Yin Zheng, Mingkui Tan, Qi Chen, Jian Chen, Peilin Zhao, and Junzhou Huang. Nat: Neural architecture transformer for accurate and compact architectures.Advances in Neural Information Processing Systems, 32, 2019

2019

[16] [16]

arXiv preprint arXiv:2506.09018 , year=

Marton Havasi, Brian Karrer, Itai Gat, and Ricky TQ Chen. Edit flows: Flow matching with edit operations. arXiv preprint arXiv:2506.09018, 2025

work page arXiv 2025

[17] [17]

Simulating 500 million years of evolution with a language model

TomasHayes, RoshanRao, HalilAkin, NicholasJSofroniew, DenizOktay, ZemingLin, RobertVerkuil, VincentQ Tran, Jonathan Deaton, Marius Wiggert, et al. Simulating 500 million years of evolution with a language model. bioRxiv, pages 2024–07, 2024

2024

[18] [18]

Diffusionbert: Improving gen- erative masked language models with diffusion models

Zhengfu He, Tianxiang Sun, Kuanning Wang, Xuanjing Huang, and Xipeng Qiu. Diffusionbert: Improving gen- erative masked language models with diffusion models. InAnnual Meeting of the Association for Computational Linguistics, 2023

2023

[19] [19]

Denoising diffusion probabilistic models

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020. URLhttps://proceedings.neurips.cc/paper/2020/ file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf

2020

[20] [20]

Argmax flows and multinomial diffusion: Learning categorical distributions

Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34:12454– 12465, 2021

2021

[21] [21]

Elucidatingthedesignspaceofmultimodalproteinlanguagemodels

Cheng-Yen Hsieh, Xinyou Wang, Daiheng Zhang, Dongyu Xue, Fei Ye, Shujian Huang, Zaixiang Zheng, and QuanquanGu. Elucidatingthedesignspaceofmultimodalproteinlanguagemodels. In Forty-secondInternational Conference on Machine Learning, 2025

2025

[22] [22]

Learning inverse folding from millions of predicted structures

Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer, and Alexander Rives. Learning inverse folding from millions of predicted structures. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors,Proceedings of the 39th International Conference on Machine Learning, volume 162 ofP...

2022

[23] [23]

Gemme: a simple and fast global epistatic model predicting mutational effects.Molecular biology and evolution, 36(11):2604–2619, 2019

Elodie Laine, Yasaman Karami, and Alessandra Carbone. Gemme: a simple and fast global epistatic model predicting mutational effects.Molecular biology and evolution, 36(11):2604–2619, 2019

2019

[24] [24]

Language models of protein sequences at the scale of evolution enable accurate structure prediction.BioRxiv, 2022

Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, et al. Language models of protein sequences at the scale of evolution enable accurate structure prediction.BioRxiv, 2022

2022

[25] [25]

Evolutionary-scale prediction of atomic-level protein structure with a language model

Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637):1123–1130, 2023

2023

[26] [26]

arXiv preprint arXiv:2509.24007 , year=

Yangzhou Liu, Yue Cao, Hao Li, Gen Luo, Zhe Chen, Weiyun Wang, Xiaobo Liang, Biqing Qi, Lijun Wu, Changyao Tian, et al. Sequential diffusion language models.arXiv preprint arXiv:2509.24007, 2025

work page arXiv 2025

[27] [27]

Expert-guided pro- tein language models enable accurate and blazingly fast fitness prediction.Bioinformatics, 40(11):btae621, 11

Céline Marquet, Julius Schlensok, Marina Abakarova, Burkhard Rost, and Elodie Laine. Expert-guided pro- tein language models enable accurate and blazingly fast fitness prediction.Bioinformatics, 40(11):btae621, 11

[28] [28]

doi: 10.1093/bioinformatics/btae621

ISSN 1367-4811. doi: 10.1093/bioinformatics/btae621. URLhttps://doi.org/10.1093/bioinformatics/ btae621

work page doi:10.1093/bioinformatics/btae621

[29] [29]

Language models enable zero-shot prediction of the effects of mutations on protein function

Joshua Meier, Roshan Rao, Robert Verkuil, Jason Liu, Tom Sercu, and Alex Rives. Language models enable zero-shot prediction of the effects of mutations on protein function. InAdvancesin Neural Information Processing Systems, pages 29287–29303, 2021

2021

[30] [30]

Dima: Diffusion mamba – a diffusion model with state space backbone for protein design

Alexey Meshchaninov, Daniil Zinchenko, Andrey Golovin, Sergey Evfratov, Alexey Chertkov, and Nikita Nikitin. Dima: Diffusion mamba – a diffusion model with state space backbone for protein design. arXiv preprint arXiv:2410.13514, 2024

work page arXiv 2024

[31] [31]

Transforming the language of life: transformer neural networks for protein prediction tasks

Ananthan Nambiar, Maeve Heflin, Simon Liu, Sergei Maslov, Mark Hopkins, and Anna Ritz. Transforming the language of life: transformer neural networks for protein prediction tasks. In Proceedings of the 11th ACM international conference on bioinformatics, computational biology and health informatics, pages 1–8, 2020. 14

2020

[32] [32]

Scaling up masked diffusion models on text

Shen Nie, Fengqi Zhu, Chao Du, Tianyu Pang, Qian Liu, Guangtao Zeng, Min Lin, and Chongxuan Li. Scaling up masked diffusion models on text. InThe Thirteenth International Conference on Learning Representations, 2024

2024

[33] [33]

Large Language Diffusion Models

Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models.arXiv preprint arXiv:2502.09992, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025

[34] [34]

Progen2: exploring the boundaries of protein language models.arXiv preprint arXiv:2206.13517, 2022

Erik Nijkamp, Jeffrey Ruffolo, Eli N Weinstein, Nikhil Naik, and Ali Madani. Progen2: exploring the boundaries of protein language models.arXiv preprint arXiv:2206.13517, 2022

work page arXiv 2022

[35] [35]

Proteingym: Large-scale benchmarks for protein fitness prediction and design

Pascal Notin, Aaron W Kollasch, Daniel Ritter, Lood Van Niekerk, Steffan Paul, Han Spinner, Nathan J Rollins, Ada Shaw, Ruben Weitzman, Jonathan Frazer, et al. Proteingym: Large-scale benchmarks for protein fitness prediction and design. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023

2023

[36] [36]

Your absorbing discrete diffusion secretly models the conditional distributions of clean data

Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. InThe Thirteenth International Conference on Learning Representations, 2024

2024

[37] [37]

Evaluating protein transfer learning with tape

Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Peter Chen, John Canny, Pieter Abbeel, and Yun Song. Evaluating protein transfer learning with tape. Advances in neural information processing systems, 32, 2019

2019

[38] [38]

Diffuser: Diffusion via edit-based reconstruction

Machel Reid, Vincent Josua Hellendoorn, and Graham Neubig. Diffuser: Diffusion via edit-based reconstruction. In International Conference on Learning Representations, 2022

2022

[39] [39]

Lawrence Zitnick, Jerry Ma, and Rob Fergus

Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C. Lawrence Zitnick, Jerry Ma, and Rob Fergus. Biological structure and function emerge from scal- ing unsupervised learning to 250 million protein sequences. PNAS, 2019. doi: 10.1101/622803. URL https://www.biorxiv.org/content/10.1101/622803v4

work page doi:10.1101/622803 2019

[40] [40]

Simpleandeffectivemaskeddiffusionlanguagemodels

Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, andVolodymyrKuleshov. Simpleandeffectivemaskeddiffusionlanguagemodels. AdvancesinNeuralInformation Processing Systems, 37:130136–130184, 2024

2024

[41] [41]

The diffusion duality

Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin T Chiu, and Volodymyr Kuleshov. The diffusion duality. InForty-secondInternational Conference on Machine Learning, 2025

2025

[42] [42]

Simple guidance mechanisms for discrete diffusion models

Yair Schiff, Subham Sekhar Sahoo, Hao Phung, Guanghan Wang, Sam Boshar, Hugo Dalla-torre, Bernardo P de Almeida, Alexander M Rush, Thomas PIERROT, and Volodymyr Kuleshov. Simple guidance mechanisms for discrete diffusion models. InThe Thirteenth International Conference on Learning Representations, 2025

2025

[43] [43]

Simplified and generalized masked diffusion for discrete data.Advancesin neural information processing systems, 37:103131–103167, 2024

Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and generalized masked diffusion for discrete data.Advancesin neural information processing systems, 37:103131–103167, 2024

2024

[44] [44]

Deep unsupervised learning using nonequilibrium thermodynamics

Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Francis Bach and David Blei, editors,International Conference on Machine Learning, volume 37 ofProceedings of Machine Learning Research, pages 2256–2265, Lille, France, 07–09 Jul

[45] [45]

URLhttps://proceedings.mlr.press/v37/sohl-dickstein15.html

PMLR, PMLR. URLhttps://proceedings.mlr.press/v37/sohl-dickstein15.html

[46] [46]

Denoising diffusion implicit models

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2020

2020

[47] [47]

Score- based generative modeling through stochastic differential equations

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score- based generative modeling through stochastic differential equations. InInternational Conference on Learning Representations, 2020

2020

[48] [48]

Saprot: Protein language modeling with structure-aware vocabulary.bioRxiv, pages 2023–10, 2023

Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. Saprot: Protein language modeling with structure-aware vocabulary.bioRxiv, pages 2023–10, 2023

2023

[49] [49]

PoET: A generative model of protein families as sequences-of-sequences

Timothy F Truong Jr and Tristan Bepler. PoET: A generative model of protein families as sequences-of-sequences. In Advancesin Neural Information Processing Systems, 2024

2024

[50] [50]

Language models generalize beyond natural proteins.bioRxiv, pages 2022–12, 2022

Robert Verkuil, Ori Kabeli, Yilun Du, Basile IM Wicky, Lukas F Milles, Justas Dauparas, David Baker, Sergey Ovchinnikov, Tom Sercu, and Alexander Rives. Language models generalize beyond natural proteins.bioRxiv, pages 2022–12, 2022. 15

2022

[51] [51]

Gen- eralized interpolating discrete diffusion

Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, and Thomas Hofmann. Gen- eralized interpolating discrete diffusion. InForty-secondInternational Conference on Machine Learning, 2025

2025

[52] [52]

arXiv preprint arXiv:2410.13782 , year=

Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, and Quanquan Gu. Dplm-2: A multimodal diffusion protein language model.arXiv preprint arXiv:2410.13782, 2024

work page arXiv 2024

[53] [53]

arXiv preprint arXiv:2402.18567 , year=

Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, and Quanquan Gu. Diffusion language models are versatile protein learners.arXiv preprint arXiv:2402.18567, 2024

work page arXiv 2024

[54] [54]

Dreamon: Diffusion language models for code infilling beyond fixed-size canvas, 2025

Zirui Wu, Lin Zheng, Zhihui Xie, Jiacheng Ye, Jiahui Gao, Yansong Feng, Zhenguo Li, Victoria W., Guorui Zhou, and Lingpeng Kong. Dreamon: Diffusion language models for code infilling beyond fixed-size canvas, 2025. URL https://hkunlp.github.io/blog/2025/dreamon

2025

[55] [55]

Modeling protein using large-scale pretrain language model

Yijia Xiao, Jiezhong Qiu, Ziang Li, Chang-Yu Hsieh, and Jie Tang. Modeling protein using large-scale pretrain language model. arXiv preprint arXiv:2108.07435, 2021

work page arXiv 2021

[56] [56]

Convolutions are competitive with transformers for protein sequence pretraining

Kevin K Yang, Alex X Lu, and Nicolo Fusi. Convolutions are competitive with transformers for protein sequence pretraining. bioRxiv, pages 2022–05, 2022

2022

[57] [57]

Dream 7B: Diffusion Large Language Models

Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b: Diffusion large language models.arXiv preprint arXiv:2508.15487, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025

[58] [58]

arXiv preprint arXiv:2308.12219 , year =

Jiasheng Ye, Zaixiang Zheng, Yu Bao, Lihua Qian, and Quanquan Gu. Diffusion language models can perform many tasks with scaling and instruction-finetuning.arXiv preprint arXiv:2308.12219, 2023

work page arXiv 2023

[59] [59]

arXiv preprint arXiv:2302.10025 , year=

Jiasheng Ye, Zaixiang Zheng, Yu Bao, Lihua Qian, and Mingxuan Wang. Dinoiser: Diffused conditional sequence learning by manipulating noises.arXiv preprint arXiv:2302.10025, 2023

work page arXiv 2023

[60] [60]

A reparameterized discrete diffusion model for text generation.arXiv preprint arXiv:2302.05737, 2023

Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong. A reparameterized discrete diffusion model for text generation. arXiv preprint arXiv:2302.05737, 2023

work page arXiv 2023

[61] [61]

Structure-informed language models are protein designers

Zaixiang Zheng, Yifan Deng, Dongyu Xue, Yi Zhou, Fei YE, and Quanquan Gu. Structure-informed language models are protein designers. InInternational Conference on Machine Learning, 2023. 16 Appendix A Training Details. A.1 Substitution Learning with Contextualized Evolutionary Noise The quality of the DPLM-Evo heavily depends on how the substitution proces...

2023