Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems

Colin Raffel; Daniel P. W. Ellis

Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1512.08756 v5 pith:ASI366PQ submitted 2015-12-29 cs.LG cs.NE

Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems

Colin Raffel , Daniel P. W. Ellis This is my paper

classification cs.LG cs.NE

keywords attentionfeed-forwardlong-termmemorymodelnetworksproblemssolve

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

0 comments

read the original abstract

We propose a simplified model of attention which is applicable to feed-forward neural networks and demonstrate that the resulting model can solve the synthetic "addition" and "multiplication" long-term memory problems for sequence lengths which are both longer and more widely varying than the best published results for these tasks.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

An Attention Mechanism for Musical Instrument Recognition
cs.IR 2019-07 unverdicted novelty 5.0

An attention model improves multi-label instrument recognition accuracy on the weakly labeled OpenMIC dataset compared to baseline, RNN, and fully connected networks across 20 instruments.