Reverse Engineering Configurations of Neural Text Generation Models

Andrew Tomkins; Che Zheng; Clifford Brunk; Dara Bahri; Donald Metzler; Yi Tay

Reverse Engineering Configurations of Neural Text Generation Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2004.06201 v1 pith:22AESH3A submitted 2020-04-13 cs.CL cs.IRcs.LG

Reverse Engineering Configurations of Neural Text Generation Models

Yi Tay , Dara Bahri , Che Zheng , Clifford Brunk , Donald Metzler , Andrew Tomkins This is my paper

classification cs.CL cs.IRcs.LG

keywords textartifactschoicesgeneratedmodelingmodelsneuralmodel

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

0 comments

read the original abstract

This paper seeks to develop a deeper understanding of the fundamental properties of neural text generations models. The study of artifacts that emerge in machine generated text as a result of modeling choices is a nascent research area. Previously, the extent and degree to which these artifacts surface in generated text has not been well studied. In the spirit of better understanding generative text models and their artifacts, we propose the new task of distinguishing which of several variants of a given model generated a piece of text, and we conduct an extensive suite of diagnostic tests to observe whether modeling choices (e.g., sampling methods, top-$k$ probabilities, model architectures, etc.) leave detectable artifacts in the text they generate. Our key finding, which is backed by a rigorous set of experiments, is that such artifacts are present and that different modeling choices can be inferred by observing the generated text alone. This suggests that neural text generators may be more sensitive to various modeling choices than previously thought.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

DSIPA: Detecting LLM-Generated Texts via Sentiment-Invariant Patterns Divergence Analysis
cs.CL 2026-04 unverdicted novelty 6.0

DSIPA is a zero-shot black-box detector that uses sentiment distribution consistency and preservation metrics to identify LLM text, reporting up to 49.89% F1 gains over baselines across domains and models.