Pith. sign in

REVIEW 4 cited by

Improving Neural Language Models with a Continuous Cache

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1612.04426 v1 pith:SPEWVZYA submitted 2016-12-13 cs.CL cs.LG

Improving Neural Language Models with a Continuous Cache

Edouard Grave , Armand Joulin , Nicolas Usunier This is my paper
classification cs.CL cs.LG
keywords memorylanguagemodelsneuralaugmentedcachehiddenmodel
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

We propose an extension to neural network language models to adapt their prediction to the recent history. Our model is a simplified version of memory augmented networks, which stores past hidden activations as memory and accesses them through a dot product with the current hidden activation. This mechanism is very efficient and scales to very large memory sizes. We also draw a link between the use of external memory in neural network and cache models used with count based language models. We demonstrate on several language model datasets that our approach performs significantly better than recent memory augmented networks.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. On the Importance and Evaluation of Narrativity in Natural Language AI Explanations

    cs.CL 2026-04 unverdicted novelty 6.0

    XAI explanations should be narratives with continuous structure, cause-effect, fluency and diversity, and new metrics are needed to evaluate this better than standard NLP scores.

  2. Compressive Transformers for Long-Range Sequence Modelling

    cs.LG 2019-11 unverdicted novelty 6.0

    Compressive Transformer sets new records on WikiText-103 (17.1 ppl) and Enwik8 (0.97 bpc) via memory compression and introduces the PG-19 long-range language benchmark.

  3. Parallel Causal Associative Fields: Gated Sparse Memory for Long-Context Language Modeling

    cs.LG 2026-06 unverdicted novelty 5.0

    PCAF uses parallel hash-based associative memory over causal records plus a learned gate to reach 36.31 perplexity on WikiText-103 and 52.45 on PG-19 at 303M parameters and 2048 context while running at 0.61M tokens/s...

  4. ARMIN: Towards a More Efficient and Light-weight Recurrent Memory Network

    cs.LG 2019-06 unverdicted novelty 5.0

    ARMIN introduces auto-addressing via hidden states and a novel RNN cell to produce a lighter recurrent memory network with lower overhead than existing MANNs or vanilla LSTMs.