On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
read the original abstract
Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates a correct translation from this representation. In this paper, we focus on analyzing the properties of the neural machine translation using two models; RNN Encoder--Decoder and a newly proposed gated recursive convolutional neural network. We show that the neural machine translation performs relatively well on short sentences without unknown words, but its performance degrades rapidly as the length of the sentence and the number of unknown words increase. Furthermore, we find that the proposed gated recursive convolutional network learns a grammatical structure of a sentence automatically.
This paper has not been read by Pith yet.
Forward citations
Cited by 42 Pith papers
-
Session-based Recommendations with Recurrent Neural Networks
RNNs with ranking loss outperform item-to-item baselines for session-based recommendations on two datasets.
-
Streaming Reinforcement Learning under Partial Observability with Real-Time Recurrent Learning
Recurrent trace units enable exact RTRL with linear time/memory for streaming RL under partial observability, sustaining performance on long-chain memory tasks where TBPTT baselines collapse.
-
Weakly Supervised Cross-Modal Learning for 4D Radar Scene Flow Estimation
Weakly supervised iterative framework for radar scene flow estimation using back-projected 2D instance masks and odometry-based rigid static loss to outperform LiDAR-dependent and fully supervised baselines on the VoD...
-
Weakly Supervised Cross-Modal Learning for 4D Radar Scene Flow Estimation
A task-specific iterative framework for weakly supervised 4D radar scene flow estimation uses instance-aware self-supervised losses from 2D tracking/segmentation and a rigid static loss from odometry to outperform LiD...
-
PluRule: A Benchmark for Moderating Pluralistic Communities on Social Media
PluRule is a new multimodal multilingual benchmark showing that state-of-the-art vision-language models perform only marginally better than a trivial baseline at detecting specific rule violations in pluralistic onlin...
-
TCD-Arena: Assessing Robustness of Time Series Causal Discovery Methods Against Assumption Violations
TCD-Arena is a new customizable testing framework that runs millions of experiments to map how 33 different assumption violations affect time series causal discovery methods and shows ensembles can boost overall robustness.
-
Reduced-Order Surrogates for Forced Flexible Mesh Coastal-Ocean Models
Koopman autoencoders with forcings and temporal unrolling deliver accurate year-long predictions for coastal-ocean models at 300-1400x speedup, outperforming POD in two of three cases.
-
Recurrent Video Masked Autoencoders
RVM uses recurrent computation inside a masked autoencoder to learn video representations that match or exceed prior video and image models on classification, tracking, and dense spatial tasks with up to 30x better pa...
-
Estimation--Prediction Tradeoff in Causal Probabilistic Temporal Graphs
Characterizes an estimation-prediction tradeoff in binary logistic models for causal probabilistic temporal graphs and proposes a framework to jointly evaluate temporal link prediction with causal parameter recovery v...
-
Conserved Kinematic Representations enable Zero-Shot Decoding in Handwriting BCIs
A zero-shot machine learning decoder for handwriting BCIs achieves 64% hits@3 retrieval on unseen letters by exploiting conserved kinematic neural representations.
-
Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balance
Stable-GFlowNet stabilizes GFN training for LLM red-teaming by eliminating Z estimation via pairwise comparisons and robust masking against noisy rewards while adding a fluency stabilizer.
-
Neural architectures for resolving references in program code
New seq2seq architectures for permutation indexing outperform baselines on synthetic reference-resolution tasks and reduce real decompilation error rates by 42%.
-
The illusory simplicity of the feedforward pass: evidence for the dynamical nature of stimulus encoding along the primate ventral stream
Primate ventral stream encodes visual stimuli through evolving neural dynamics that carry category information beyond any fixed spatial pattern during the initial feedforward pass.
-
Leveraging Artist Catalogs for Cold-Start Music Recommendation
ACARec attends over artist catalogs to generate CF embeddings for new tracks, more than doubling recall and NDCG versus content-only baselines in music recommendation.
-
M$^2$RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling
M²RNN achieves perfect state tracking at unseen lengths and outperforms Gated DeltaNet hybrids by 0.4-0.5 perplexity on 7B models with 3x smaller recurrent states.
-
SpectraLLM: Uncovering the Ability of LLMs for Molecular Structure Elucidation from Multi-Spectral Data
SpectraLLM is an LLM fine-tuned to predict small-molecule structures from single or multiple spectra, reporting state-of-the-art results on four public benchmarks with gains from multi-modal input.
-
DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectory
DragNUWA integrates text, image, and trajectory controls into a diffusion video model using a Trajectory Sampler, Multiscale Fusion, and Adaptive Training to enable fine-grained open-domain video generation.
-
Variational Context: Exploiting Visual and Textual Context for Grounding Referring Expressions
A variational Bayesian framework exploits reciprocity between referents and context plus semantic reproduction to improve referring expression grounding over pairwise methods in supervised and unsupervised settings.
-
Creating A Neural Pedagogical Agent by Jointly Learning to Review and Assess
Bidirectional RNN with attention models real-time user knowledge from question-response sequences to predict correctness, outperforming baselines especially for new users on a large TOEIC mobile app dataset.
-
Recurrent Adversarial Service Times
RNN for arrivals paired with recurrent GAN for service times to model queuing dynamics without assuming specific inter-event distributions.
-
Learning Belief Representations for Imitation Learning in POMDPs
BMIL learns belief modules jointly with policies for GAIL-style imitation learning in POMDPs, outperforming separate training and standard GAIL on continuous control tasks.
-
Leveraging Multimodality for Real-Time Classification of Transients and Variables found by the Zwicky Transient Facility
ORACLE-2 multimodal classifiers raise macro F1 from 0.52-0.66 (light-curve only) to 0.73 on ZTF Bright Transient Survey data and reach 0.88 on simulated ELAsTiCC data.
-
Direct Advantage Estimation for Scalable and Sample-efficient Deep Reinforcement Learning
Extends DAE theory to POMDPs with minimal changes and introduces discrete latent dynamics to cut computational cost, with ALE experiments showing scalability and retained sample efficiency.
-
Linear Recurrent Unit with Semantic Modulation for Image Super-Resolution
Introduces an LRU-based network with semantic modulation that claims to outperform prior super-resolution methods at similar computational cost.
-
Probabilistic Verification of Recurrent Neural Networks for Single and Multi-Agent Reinforcement Learning
RNN-ProVe uses policy-driven sampling and statistical error bounds to produce high-confidence probabilistic estimates of behavioral violations in RNN policies for single- and multi-agent POMDPs.
-
EHR-RAGp: Retrieval-Augmented Prototype-Guided Foundation Model for Electronic Health Records
EHR-RAGp is a retrieval-augmented EHR foundation model that employs prototype-guided retrieval to dynamically integrate relevant historical patient context, outperforming prior models on clinical prediction tasks.
-
Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balance
Stable-GFlowNet improves training stability and attack diversity in LLM red-teaming by eliminating Z estimation via contrastive trajectory balance while preserving GFN optimality.
-
Delta6: A Low-Cost, 6-DOF Force-Sensing Flexible End-Effector
Delta6 delivers a low-cost 6-DOF force-sensing end-effector with 3.8% FS accuracy using sequence models, validated on robot-arm tasks like buffing and tight assembly.
-
On Safer Reinforcement Learning for Sedation and Analgesia in Intensive Care
Offline RL for ICU sedation shows that adding 30-day mortality to the objective yields policies whose clinician agreement correlates negatively with mortality, unlike pain-only versions.
-
StateX: Enhancing RNN Recall via Post-training State Expansion
StateX post-trains RNNs to expand recurrent state size, improving recall and in-context learning with negligible parameter growth.
-
From Time-series Generation, Model Selection to Transfer Learning: A Comparative Review of Pixel-wise Approaches for Large-scale Crop Mapping
A comparative review with experiments identifying optimal preprocessing, models, and transfer strategies for large-scale pixel-wise crop mapping using Landsat 8 data across five sites.
-
Enriching and Controlling Global Semantics for Text Summarization
A normalizing-flow neural topic model plus control mechanism are added to Transformer summarizers to supply and regulate global semantics, with reported gains over prior models on five benchmarks.
-
Reasoning and Generalization in RL: A Tool Use Perspective
Proposes a tool-use inspired framework with multiple test sets to measure specified types of generalization in RL.
-
Beyond Time Series: Spatial Reasoning for Epidemic Forecasting via Multimodal Learning
M-SPICE uses attention-based multimodal fusion to integrate region-level temporal data with spatially localized auxiliary signals for improved epidemic forecasting on COVID-19, influenza, and ILI tasks.
-
Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation
G2Rec unifies holistic graph-based user co-engagement modeling with semantic tokenization for scalable generative recommendation without ground-truth user interests.
-
Fast-ULCNet: A fast and ultra low complexity network for single-channel speech enhancement
Fast-ULCNet matches original ULCNet speech enhancement quality while cutting model size by more than half and latency by 34% via FastGRNN replacement and a state-drift filter.
-
Neural Cross-Domain Collaborative Filtering with Shared Entities
NeuCDCF is a wide-and-deep neural architecture for cross-domain collaborative filtering that jointly learns matrix factorization and deep representations, reporting better performance than prior CDCF models on four re...
-
Polyphone Disambiguation for Mandarin Chinese Using Conditional Neural Network with Multi-level Embedding Features
A conditional neural network using bidirectional RNN sentence encoding and multi-level word/sentence embeddings reaches 94.69% accuracy on a public Mandarin polyphone dataset.
-
Rare Disease Detection by Sequence Modeling with Generative Adversarial Networks
A GAN-boosted RNN model reaches 0.56 PR-AUC for rare EPI detection on 1.8 million patients and outperforms benchmarks.
-
Alchemy: A Quantum Chemistry Dataset for Benchmarking AI Models
Alchemy releases a new quantum chemistry dataset of 119k molecules and 12 properties plus GNN benchmarks to support AI model development for molecular science.
-
Large Language Models: A Survey
The paper surveys key large language models, their training methods, datasets, evaluation benchmarks, and future research directions in the field.
-
Deep learning applied to computational mechanics: A comprehensive review, state of the art, and the classics
A comprehensive review of deep learning techniques for computational mechanics, including LSTM for constitutive modeling, PINNs for PDE solving, optimizers, and kernel methods.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.