Towards Human-Level Book-Writing Capability

Jan Zierstek; Matteo Batelic; Maya Medjad; Tim Sch\"onenberger

arxiv: 2605.17064 · v2 · pith:SNBJQ56Pnew · submitted 2026-05-16 · 💻 cs.AI

Towards Human-Level Book-Writing Capability

Jan Zierstek , Matteo Batelic , Maya Medjad , Tim Sch\"onenberger This is my paper

Pith reviewed 2026-06-30 19:00 UTC · model grok-4.3

classification 💻 cs.AI

keywords creative writingbook generationlong-context language modelsplanning scaffoldsupervised fine-tuningliterary stylefiction generationprompt-to-book trajectories

0 comments

The pith

Training a long-context model on inverted planning scaffolds from public-domain novels shifts its output from assistant-style prose to human literary writing and outperforms GPT-5.5 and Claude Opus 4.8.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes a new way to train language models for book-scale creative writing by turning summaries of real novels into training trajectories. Instead of fine-tuning on instructions, the model learns to expand a prompt into detailed plans and then into the full original book text. This preserves human prose as the target while making long generation feasible. The result is writing that avoids generic assistant habits and scores higher on quality evaluations than leading models. A sympathetic reader would care because current models struggle with the moral ambiguity and stylistic depth needed for fiction.

Core claim

By deriving a multi-resolution Planning Scaffold from public-domain novels through progressive summarization and then inverting the hierarchy during supervised fine-tuning, a long-context language model learns prompt-to-book generation trajectories that align its output with human literary behavior rather than instruction-following patterns, leading to superior performance on writing quality evaluations compared to GPT-5.5 and Claude Opus 4.8.

What carries the argument

The multi-resolution Planning Scaffold, created by summarizing novels at premise, chapter, and scene levels, which is inverted to train the model to expand prompts into full human-authored book text.

If this is right

The model generates stories with deception, moral ambiguity, and unreliable narration instead of avoiding them.
Generation becomes stylistically grounded in human literary behavior rather than structurally correct but generic.
Book-scale creative writing becomes learnable through this scaffold inversion without reducing to simple copying.
Purpose-built models can surpass general instruction-tuned models on creative tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the method works, it could extend to other long-form creative domains like screenplay or game narrative writing by adapting the scaffold.
Future work might test whether the intermediate summaries introduce biases that affect originality in generated content.
Applying this to non-public domain sources could raise questions about copyright in training data for creative models.
Human evaluation protocols for literary quality might need to account for the specific traits like unreliable narration that the model now produces.

Load-bearing premise

That inverting the planning scaffold from novels allows the model to learn book-scale generation while keeping the final output as unaltered human prose without the summaries biasing the style or making it mere replication.

What would settle it

A controlled human evaluation where blind readers rate the model's full book outputs higher in literary quality than those from GPT-5.5 and Claude Opus 4.8, or a failure to do so would disprove the claim.

Figures

Figures reproduced from arXiv: 2605.17064 by Jan Zierstek, Matteo Batelic, Maya Medjad, Tim Sch\"onenberger.

**Figure 2.** Figure 2: Token-level characterization of the corpus sequence. Top: upper-envelope token composi [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

**Figure 3.** Figure 3: Hierarchical structure of a composed training example. The representation begins with the [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

read the original abstract

Large language models are optimized for instruction following and agentic tasks remain poorly aligned with the requirements of high-quality creative writing. We show that a purpose-built creative writing model can outperform both GPT-5.5 and Claude Opus 4.8 on writing quality evaluation. Fiction frequently depends on behaviors that assistant-tuned models are explicitly trained to avoid, particularly deception, moral ambiguity, and unreliable narration. As a result, generated stories often appear structurally correct while remaining stylistically generic, overly explanatory, or weakly grounded in human literary behavior. We present a dataset construction and training framework for book-scale creative writing that reframes supervised fine-tuning as a prompt-to-book generation task grounded in human-authored fiction. Starting from public-domain novels, we derive a multi-resolution Planning Scaffold by summarizing each book at progressively finer levels, from a high-level premise to chapter- and scene-level structure. We then invert this hierarchy during training: the model learns to expand a prompt into increasingly detailed plans and finally into the original human-authored book text. This formulation preserves human prose as the final supervised target while using intermediate summaries to make book-scale generation learnable. We train a long-context language model on these prompt-to-book trajectories and show that this objective shifts generation away from assistant-style prose and toward human literary writing.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper outlines a coherent scaffold-inversion method for book-scale creative writing but its claims of outperforming GPT-5.5 and Claude Opus 4.8 rest on zero visible evidence or metrics.

read the letter

The main thing to know is that this paper describes a dataset construction where public-domain novels are turned into multi-resolution planning scaffolds, then inverted so a long-context model learns to expand a prompt through plans into the original human text. The goal is to shift output away from assistant-style prose toward literary fiction that includes ambiguity and unreliable narration.

The formulation is new in its specific use of the inverted hierarchy to keep human prose as the terminal target while making book-length generation trainable. It does a clear job laying out why standard instruction tuning clashes with fiction needs and how the scaffold approach tries to bridge that.

The soft spots are the performance claims. The abstract states the model beats GPT-5.5 and Claude Opus 4.8 on writing quality, yet supplies no metrics, no evaluation method, no baselines, and no analysis. That leaves the central result uncheckable. It is also unclear whether the intermediate summaries add unintended biases or whether the method scales without artifacts at full book length.

This is for people working on long-form generation and creative applications of language models. A reader who wants to test alternative fine-tuning objectives for fiction would find the method description useful if the full paper includes the missing evaluations. It deserves a serious referee because the training setup is specific enough to review and the targeted limitation is real, even though the evidence presented so far is thin.

Referee Report

2 major / 0 minor

Summary. The paper claims to introduce a dataset construction and training framework for book-scale creative writing: public-domain novels are summarized into a multi-resolution Planning Scaffold (high-level premise down to chapter/scene level); this hierarchy is inverted during supervised fine-tuning so a long-context LM learns to expand a prompt into plans and finally the original human-authored text. The resulting model is asserted to shift generation away from assistant-style prose toward human literary writing and to outperform GPT-5.5 and Claude Opus 4.8 on writing-quality evaluation.

Significance. If the claimed performance gains and stylistic shift are demonstrated with rigorous evaluation, the work would offer a concrete, scalable route to align LLMs with the specific demands of long-form fiction (deception, moral ambiguity, unreliable narration) that standard instruction tuning suppresses. The use of human-authored terminal targets and an explicit planning scaffold could also serve as a template for other domains requiring extended coherent generation.

major comments (2)

[Abstract] Abstract: the central claim that the trained model 'outperforms both GPT-5.5 and Claude Opus 4.8 on writing quality evaluation' is unsupported by any reported metrics, baselines, human or automatic evaluation protocol, statistical significance tests, or error analysis. This absence renders the primary empirical assertion unevaluable.
[Abstract] Abstract (paragraph on dataset construction and training framework): the assertion that inverting the multi-resolution Planning Scaffold 'preserves human prose as the final supervised target while using intermediate summaries to make book-scale generation learnable' is presented without ablation studies, bias analysis, or comparison to direct book-level fine-tuning, leaving open whether the scaffold introduces unintended stylistic artifacts or collapses to standard next-token prediction.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive comments on the abstract. We address each major comment below, indicating where revisions will be made to the manuscript.

read point-by-point responses

Referee: [Abstract] Abstract: the central claim that the trained model 'outperforms both GPT-5.5 and Claude Opus 4.8 on writing quality evaluation' is unsupported by any reported metrics, baselines, human or automatic evaluation protocol, statistical significance tests, or error analysis. This absence renders the primary empirical assertion unevaluable.

Authors: We agree that the abstract states this performance claim without any supporting metrics, protocols, or analysis in the manuscript. The provided text contains no evaluation results, baselines, or statistical details. We will revise the abstract to remove the specific claim of outperforming GPT-5.5 and Claude Opus 4.8, as it cannot be substantiated from the current content. The revised abstract will focus on the training framework and the described shift toward human literary writing. revision: yes
Referee: [Abstract] Abstract (paragraph on dataset construction and training framework): the assertion that inverting the multi-resolution Planning Scaffold 'preserves human prose as the final supervised target while using intermediate summaries to make book-scale generation learnable' is presented without ablation studies, bias analysis, or comparison to direct book-level fine-tuning, leaving open whether the scaffold introduces unintended stylistic artifacts or collapses to standard next-token prediction.

Authors: The referee is correct that the manuscript presents this methodological assertion without ablations, bias analysis, or direct comparisons. No such studies appear in the provided text. We will revise the relevant paragraph to include a brief discussion of the design rationale for the inverted scaffold and note the absence of ablations as a limitation. However, without performing new experiments, we cannot add full ablation results or bias analyses. revision: partial

Circularity Check

0 steps flagged

No significant circularity

full rationale

The provided abstract and description outline a dataset pipeline that extracts multi-resolution summaries from external public-domain novels and inverts them to create prompt-to-book trajectories whose terminal target is the original human-authored text. This is a conventional supervised fine-tuning construction with an independent external ground truth (public-domain books); the final supervised target is not derived from model outputs or internal definitions. No equations, fitted parameters renamed as predictions, self-citations, uniqueness theorems, or ansatzes appear in the text. The claim that the objective shifts generation toward human literary writing follows directly from preserving the original prose as the training target rather than from any self-referential reduction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the domain assumption that public-domain novels represent suitable targets for human literary behavior and that the scaffold inversion makes long-form generation learnable; no free parameters or independent evidence for the new scaffold entity are provided in the abstract.

axioms (1)

domain assumption Public-domain novels provide a representative sample of human literary fiction suitable for training targets that include deception, moral ambiguity, and unreliable narration.
The framework starts from these novels to derive the scaffolds and uses their text as the final target.

invented entities (1)

Multi-resolution Planning Scaffold no independent evidence
purpose: To provide hierarchical summaries that allow the model to learn expansion from prompt to full book text.
New structure introduced in the paper to make book-scale generation learnable.

pith-pipeline@v0.9.1-grok · 5758 in / 1608 out tokens · 54254 ms · 2026-06-30T19:00:47.489133+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Reference graph

Works this paper leans on

27 extracted references · 17 canonical work pages · 10 internal anchors

[1]

InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4393–4479, Abu Dhabi, United Arab Emirates, 2022

Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein.Re3: Generating longer stories with recursive reprompting and revision. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4393–4479, Abu Dhabi, United Arab Emirates, 2022. Association for Computational Linguistics. https://aclanthology.org/2022.emnlp-main.296/

2022
[2]

InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pages 3378–3465, Toronto, Canada, 2023

Kevin Yang, Dan Klein, Nanyun Peng, and Yuandong Tian.DOC: Improving long story coherence with detailed outline control. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pages 3378–3465, Toronto, Canada, 2023. Association for Computational Linguistics. https://aclanthology.org/2023.acl-long.190/

2023
[3]

Generating long-form story using dynamic hierarchical outlining with memory-enhancement

Qianyue Wang, Jinwu Hu, Zhengping Li, Yufeng Wang, Daiyuan Li, Yu Hu, and Mingkui Tan. Generating long-form story using dynamic hierarchical outlining with memory-enhancement. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1352–1391, Albuquerq...

2025
[4]

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback....

work page internal anchor Pith review Pith/arXiv arXiv 2022
[5]

Plan-and-Write: Towards better automatic storytelling

Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. Plan-and-Write: Towards better automatic storytelling. InProceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, volume 33, number 1, pages 7378–7385, 2019. https://doi.org/10.1609/aaai.v33i01.33017378

work page doi:10.1609/aaai.v33i01.33017378 2019
[6]

Hierarchical neural story generation

Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pages 889–898, Melbourne, Australia, 2018. Association for Computational Linguistics. https://aclanthology.org/P18-1082/

2018
[7]

Strategies for structuring story generation

Angela Fan, Mike Lewis, and Yann Dauphin. Strategies for structuring story generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2650–2660, Florence, Italy, 2019. Association for Computational Linguistics. https://aclanthology.org/P19-1254/

2019
[8]

InProceedings of the 2020 15 Conference on Empirical Methods in Natural Language Processing, pages 4274–4295, Online, 2020

Hannah Rashkin, Asli Celikyilmaz, Yejin Choi, and Jianfeng Gao.PlotMachines: Outline-conditioned generation with dynamic plot state tracking. InProceedings of the 2020 15 Conference on Empirical Methods in Natural Language Processing, pages 4274–4295, Online, 2020. Association for Computational Linguistics. https://aclanthology.org/2020.emnlp-main.349/

2020
[9]

Riedl and R

Mark O. Riedl and R. Michael Young. Narrative planning: Balancing plot and character. Journal of Artificial Intelligence Research, 39:217–268, 2010. https://doi.org/10.1613/jair.2989

work page doi:10.1613/jair.2989 2010
[10]

Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, and Mark O

Lara J. Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, and Mark O. Riedl. Event representations for automated story generation with deep neural nets. InProceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pages 868–875, 2018

2018
[11]

Project Gutenberg.Project Gutenberg.https://www.gutenberg.org/
[12]

Qwen3 Technical Report

Qwen Team. Qwen3 Technical Report.arXiv preprint arXiv:2505.09388, 2025. https://arxiv.org/abs/2505.09388

work page internal anchor Pith review Pith/arXiv arXiv 2025
[13]

Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias

Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander Ratner, Ranjay Krishna, Jiaming Shen, and Chao Zhang. Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias. InAdvances in Neural Information Processing Systems, 2023. https://arxiv.org/abs/2306.15895

work page arXiv 2023
[14]

TinyStories: How Small Can Language Models Be and Still Speak Coherent English?

Ronen Eldan and Yuanzhi Li. TinyStories: How Small Can Language Models Be and Still Speak Coherent English?arXiv preprint arXiv:2305.07759, 2023. https://arxiv.org/abs/2305.07759

work page internal anchor Pith review Pith/arXiv arXiv 2023
[15]

Parameterized Synthetic Text Generation with SimpleStories.arXiv preprint arXiv:2504.09184, 2025.https://arxiv.org/abs/2504.09184

Lennart Finke, Thomas Dooms, Mat Allen, Juan Diego Rodriguez, Noa Nabeshima, and Dan Braun. Parameterized Synthetic Text Generation with SimpleStories.arXiv preprint arXiv:2504.09184, 2025.https://arxiv.org/abs/2504.09184

work page arXiv 2025
[16]

Self-Boosting Large Language Models with Synthetic Preference Data.arXiv preprint arXiv:2410.06961, 2024

Qingxiu Dong, Li Dong, Xingxing Zhang, Zhifang Sui, and Furu Wei. Self-Boosting Large Language Models with Synthetic Preference Data.arXiv preprint arXiv:2410.06961, 2024. https://arxiv.org/abs/2410.06961

work page arXiv 2024
[17]

Alexander H. Liu, Kartik Khandelwal, Sandeep Subramanian, Victor Jouault, Abhinav Rastogi, Adrien Sadé, Alan Jeffares, Albert Jiang, Alexandre Cahill, Alexandre Gavaudan, Alexandre Sablayrolles, Amélie Héliou, Amos You, Andy Ehrenberg, Andy Lo, Anton Eliseev, Antonia Calvi, Avinash Sooriyarachchi, Baptiste Bout, Baptiste Rozière, and others. Ministral 3.a...

work page internal anchor Pith review Pith/arXiv arXiv 2026
[18]

Ministral 3 14B Base

Mistral AI. Ministral 3 14B Base. Hugging Face model card, 2025. https://huggingface.co/mistralai/Ministral-3-14B-Base-2512

2025
[19]

Training Verifiers to Solve Math Word Problems

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021.https://arxiv.org/abs/2110.14168

work page internal anchor Pith review Pith/arXiv arXiv 2021
[20]

The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generation.arXiv preprint arXiv:2412.04318, 2024

Fredrik Carlsson, Fangyu Liu, Daniel Ward, Murathan Kurfali, and Joakim Nivre. The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generation.arXiv preprint arXiv:2412.04318, 2024. https://arxiv.org/abs/2412.04318

work page arXiv 2024
[21]

JAX: composable transformations of Python+NumPy programs

James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs. Software, 2018. https://github.com/google/jax. 16

2018
[22]

A Study of BFLOAT16 for Deep Learning Training

Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das, Kunal Banerjee, Sasikanth Avancha, Dharma Teja V ooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, Jiyan Yang, Jongsoo Park, Alexander Heinecke, Evangelos Georganas, Sudarshan Srinivasan, Abhisek Kundu, Misha Smelyanskiy, Bharat Kaul, and Pradeep Dubey. A study of bfloat16 for d...

work page internal anchor Pith review Pith/arXiv arXiv 1905
[23]

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. ZeRO: Memory optimizations toward training trillion parameter models.arXiv preprint arXiv:1910.02054, 2020.https://arxiv.org/abs/1910.02054

work page internal anchor Pith review Pith/arXiv arXiv 1910
[24]

Ring Attention with Blockwise Transformers for Near-Infinite Context

Hao Liu, Matei Zaharia, and Pieter Abbeel. Ring Attention with Blockwise Transformers for Near-Infinite Context.arXiv preprint arXiv:2310.01889, 2023. https://arxiv.org/abs/2310.01889

work page internal anchor Pith review Pith/arXiv arXiv 2023
[25]

Muon is Scalable for LLM Training

Jingyuan Liu, Jianlin Su, Xingcheng Yao, Zhejun Jiang, Guokun Lai, Yulun Du, Yidao Qin, Weixin Xu, Enzhe Lu, Junjie Yan, Yanru Chen, Huabin Zheng, Yibo Liu, Shaowei Liu, Bohong Yin, Weiran He, Han Zhu, Yuzhi Wang, Jianzhou Wang, Mengnan Dong, Zheng Zhang, Yongsheng Kang, Hao Zhang, Xinran Xu, Yutao Zhang, Yuxin Wu, Xinyu Zhou, and Zhilin Yang. Muon is sca...

work page internal anchor Pith review Pith/arXiv arXiv 2025
[26]

Stochastic Rounding for LLM Training: Theory and Practice

Kaan Ozkara, Tao Yu, and Youngsuk Park. Stochastic Rounding for LLM Training: Theory and Practice. InProceedings of the International Conference on Artificial Intelligence and Statistics, 2025.https://arxiv.org/abs/2502.20566

work page arXiv 2025
[27]

The Llama 3 Herd of Models

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and others. The Llama 3 Herd of Models. arXiv preprint arXiv:2407....

work page internal anchor Pith review Pith/arXiv arXiv 2024

[1] [1]

InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4393–4479, Abu Dhabi, United Arab Emirates, 2022

Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein.Re3: Generating longer stories with recursive reprompting and revision. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4393–4479, Abu Dhabi, United Arab Emirates, 2022. Association for Computational Linguistics. https://aclanthology.org/2022.emnlp-main.296/

2022

[2] [2]

InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pages 3378–3465, Toronto, Canada, 2023

Kevin Yang, Dan Klein, Nanyun Peng, and Yuandong Tian.DOC: Improving long story coherence with detailed outline control. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pages 3378–3465, Toronto, Canada, 2023. Association for Computational Linguistics. https://aclanthology.org/2023.acl-long.190/

2023

[3] [3]

Generating long-form story using dynamic hierarchical outlining with memory-enhancement

Qianyue Wang, Jinwu Hu, Zhengping Li, Yufeng Wang, Daiyuan Li, Yu Hu, and Mingkui Tan. Generating long-form story using dynamic hierarchical outlining with memory-enhancement. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1352–1391, Albuquerq...

2025

[4] [4]

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback....

work page internal anchor Pith review Pith/arXiv arXiv 2022

[5] [5]

Plan-and-Write: Towards better automatic storytelling

Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. Plan-and-Write: Towards better automatic storytelling. InProceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, volume 33, number 1, pages 7378–7385, 2019. https://doi.org/10.1609/aaai.v33i01.33017378

work page doi:10.1609/aaai.v33i01.33017378 2019

[6] [6]

Hierarchical neural story generation

Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pages 889–898, Melbourne, Australia, 2018. Association for Computational Linguistics. https://aclanthology.org/P18-1082/

2018

[7] [7]

Strategies for structuring story generation

Angela Fan, Mike Lewis, and Yann Dauphin. Strategies for structuring story generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2650–2660, Florence, Italy, 2019. Association for Computational Linguistics. https://aclanthology.org/P19-1254/

2019

[8] [8]

InProceedings of the 2020 15 Conference on Empirical Methods in Natural Language Processing, pages 4274–4295, Online, 2020

Hannah Rashkin, Asli Celikyilmaz, Yejin Choi, and Jianfeng Gao.PlotMachines: Outline-conditioned generation with dynamic plot state tracking. InProceedings of the 2020 15 Conference on Empirical Methods in Natural Language Processing, pages 4274–4295, Online, 2020. Association for Computational Linguistics. https://aclanthology.org/2020.emnlp-main.349/

2020

[9] [9]

Riedl and R

Mark O. Riedl and R. Michael Young. Narrative planning: Balancing plot and character. Journal of Artificial Intelligence Research, 39:217–268, 2010. https://doi.org/10.1613/jair.2989

work page doi:10.1613/jair.2989 2010

[10] [10]

Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, and Mark O

Lara J. Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, and Mark O. Riedl. Event representations for automated story generation with deep neural nets. InProceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pages 868–875, 2018

2018

[11] [11]

Project Gutenberg.Project Gutenberg.https://www.gutenberg.org/

[12] [12]

Qwen3 Technical Report

Qwen Team. Qwen3 Technical Report.arXiv preprint arXiv:2505.09388, 2025. https://arxiv.org/abs/2505.09388

work page internal anchor Pith review Pith/arXiv arXiv 2025

[13] [13]

Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias

Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander Ratner, Ranjay Krishna, Jiaming Shen, and Chao Zhang. Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias. InAdvances in Neural Information Processing Systems, 2023. https://arxiv.org/abs/2306.15895

work page arXiv 2023

[14] [14]

TinyStories: How Small Can Language Models Be and Still Speak Coherent English?

Ronen Eldan and Yuanzhi Li. TinyStories: How Small Can Language Models Be and Still Speak Coherent English?arXiv preprint arXiv:2305.07759, 2023. https://arxiv.org/abs/2305.07759

work page internal anchor Pith review Pith/arXiv arXiv 2023

[15] [15]

Parameterized Synthetic Text Generation with SimpleStories.arXiv preprint arXiv:2504.09184, 2025.https://arxiv.org/abs/2504.09184

Lennart Finke, Thomas Dooms, Mat Allen, Juan Diego Rodriguez, Noa Nabeshima, and Dan Braun. Parameterized Synthetic Text Generation with SimpleStories.arXiv preprint arXiv:2504.09184, 2025.https://arxiv.org/abs/2504.09184

work page arXiv 2025

[16] [16]

Self-Boosting Large Language Models with Synthetic Preference Data.arXiv preprint arXiv:2410.06961, 2024

Qingxiu Dong, Li Dong, Xingxing Zhang, Zhifang Sui, and Furu Wei. Self-Boosting Large Language Models with Synthetic Preference Data.arXiv preprint arXiv:2410.06961, 2024. https://arxiv.org/abs/2410.06961

work page arXiv 2024

[17] [17]

Alexander H. Liu, Kartik Khandelwal, Sandeep Subramanian, Victor Jouault, Abhinav Rastogi, Adrien Sadé, Alan Jeffares, Albert Jiang, Alexandre Cahill, Alexandre Gavaudan, Alexandre Sablayrolles, Amélie Héliou, Amos You, Andy Ehrenberg, Andy Lo, Anton Eliseev, Antonia Calvi, Avinash Sooriyarachchi, Baptiste Bout, Baptiste Rozière, and others. Ministral 3.a...

work page internal anchor Pith review Pith/arXiv arXiv 2026

[18] [18]

Ministral 3 14B Base

Mistral AI. Ministral 3 14B Base. Hugging Face model card, 2025. https://huggingface.co/mistralai/Ministral-3-14B-Base-2512

2025

[19] [19]

Training Verifiers to Solve Math Word Problems

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021.https://arxiv.org/abs/2110.14168

work page internal anchor Pith review Pith/arXiv arXiv 2021

[20] [20]

The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generation.arXiv preprint arXiv:2412.04318, 2024

Fredrik Carlsson, Fangyu Liu, Daniel Ward, Murathan Kurfali, and Joakim Nivre. The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generation.arXiv preprint arXiv:2412.04318, 2024. https://arxiv.org/abs/2412.04318

work page arXiv 2024

[21] [21]

JAX: composable transformations of Python+NumPy programs

James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs. Software, 2018. https://github.com/google/jax. 16

2018

[22] [22]

A Study of BFLOAT16 for Deep Learning Training

Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das, Kunal Banerjee, Sasikanth Avancha, Dharma Teja V ooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, Jiyan Yang, Jongsoo Park, Alexander Heinecke, Evangelos Georganas, Sudarshan Srinivasan, Abhisek Kundu, Misha Smelyanskiy, Bharat Kaul, and Pradeep Dubey. A study of bfloat16 for d...

work page internal anchor Pith review Pith/arXiv arXiv 1905

[23] [23]

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. ZeRO: Memory optimizations toward training trillion parameter models.arXiv preprint arXiv:1910.02054, 2020.https://arxiv.org/abs/1910.02054

work page internal anchor Pith review Pith/arXiv arXiv 1910

[24] [24]

Ring Attention with Blockwise Transformers for Near-Infinite Context

Hao Liu, Matei Zaharia, and Pieter Abbeel. Ring Attention with Blockwise Transformers for Near-Infinite Context.arXiv preprint arXiv:2310.01889, 2023. https://arxiv.org/abs/2310.01889

work page internal anchor Pith review Pith/arXiv arXiv 2023

[25] [25]

Muon is Scalable for LLM Training

Jingyuan Liu, Jianlin Su, Xingcheng Yao, Zhejun Jiang, Guokun Lai, Yulun Du, Yidao Qin, Weixin Xu, Enzhe Lu, Junjie Yan, Yanru Chen, Huabin Zheng, Yibo Liu, Shaowei Liu, Bohong Yin, Weiran He, Han Zhu, Yuzhi Wang, Jianzhou Wang, Mengnan Dong, Zheng Zhang, Yongsheng Kang, Hao Zhang, Xinran Xu, Yutao Zhang, Yuxin Wu, Xinyu Zhou, and Zhilin Yang. Muon is sca...

work page internal anchor Pith review Pith/arXiv arXiv 2025

[26] [26]

Stochastic Rounding for LLM Training: Theory and Practice

Kaan Ozkara, Tao Yu, and Youngsuk Park. Stochastic Rounding for LLM Training: Theory and Practice. InProceedings of the International Conference on Artificial Intelligence and Statistics, 2025.https://arxiv.org/abs/2502.20566

work page arXiv 2025

[27] [27]

The Llama 3 Herd of Models

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and others. The Llama 3 Herd of Models. arXiv preprint arXiv:2407....

work page internal anchor Pith review Pith/arXiv arXiv 2024