GLTR: Statistical Detection and Visualization of Generated Text
read the original abstract
The rapid improvement of language models has raised the specter of abuse of text generation systems. This progress motivates the development of simple methods for detecting generated text that can be used by and explained to non-experts. We develop GLTR, a tool to support humans in detecting whether a text was generated by a model. GLTR applies a suite of baseline statistical methods that can detect generation artifacts across common sampling schemes. In a human-subjects study, we show that the annotation scheme provided by GLTR improves the human detection-rate of fake text from 54% to 72% without any prior training. GLTR is open-source and publicly deployed, and has already been widely used to detect generated outputs
This paper has not been read by Pith yet.
Forward citations
Cited by 13 Pith papers
-
Language Models are Few-Shot Learners
GPT-3 shows that scaling an autoregressive language model to 175 billion parameters enables strong few-shot performance across diverse NLP tasks via in-context prompting without fine-tuning.
-
SV-Detect: AI-generated Text Detection with Steering Vectors
Steering vectors from frozen LM layers enable a lightweight classifier to detect machine-generated text robustly across domains, source models, and editing attacks.
-
SoK: Exposing the Generation and Detection Gaps in LLM-Generated Phishing
This SoK paper introduces a nine-stage taxonomy for LLM guardrail breaches in phishing, characterizes evasion and manipulation tactics, and identifies a dynamic-offense versus static-defense asymmetry.
-
ExaGPT: Example-Based Machine-Generated Text Detection for Human Interpretability
ExaGPT uses span-level similarity retrieval from human and LLM datastores to detect machine-generated text while supplying the matching spans as human-interpretable evidence, achieving up to 37-point accuracy gains ov...
-
Hitting a Moving Target: Test-Time Adaptation for AI Text Detection under Continual Distribution Shift
Test-time adaptation with semi-supervised learning leverages inference-time homogeneity to maintain AI text detection performance under adversarial humanization, new LLMs, and temporal drift.
-
GigaCheck: Detecting LLM-generated Content via Object-Centric Span Localization
GigaCheck detects LLM-generated text at both document and span levels by combining fine-tuned language-model embeddings with a DETR-like architecture that treats generated intervals as detectable objects.
-
Can AI-Generated Text be Reliably Detected?
Recursive paraphrasing attacks substantially lower detection rates for multiple AI text detectors with only minor quality loss, while a theoretical analysis ties best-case AUROC to total variation distance between hum...
-
An Exploratory Study on LLM-Generated Code and Comments in Code Repositories
Detector proxy analysis of real repositories finds LLM-like code decreasing over time and common in tests, with more in company repos, substantial clones, stable but often ungrammatical comments, and few linked bugs.
-
Multi-Level Contextual Token Relation Modeling for Machine-Generated Text Detection
A multi-level framework that models local and global relations among token detection scores to improve machine-generated text detection with low overhead.
-
Detecting LLM-Assisted Academic Dishonesty using Keystroke Dynamics
Keystroke dynamics models outperform text-only detectors for spotting LLM-assisted academic dishonesty in practical scenarios, though performance drops under adversarial conditions.
-
A Practice Auditing Framework for Large Language Model Use: Collective Empiricism, Pseudo-Rational Cognition, and Governance of AI-Generated Content
This paper proposes a conceptual auditing framework for LLM interactions to mitigate risks from mistaking AI-generated content for empirical knowledge.
-
Findings of the Counter Turing Test: AI-Generated Text Detection
Shared task findings show near-perfect binary detection of AI-generated text but greater difficulty in attributing outputs to particular language models.
-
Findings of the Counter Turing Test: AI-Generated Text Detection
Shared task findings show F1=1.0000 for binary AI text detection and 0.9531 for model attribution using fine-tuned DeBERTa and BART transformers with ensembles.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.