REVIEW 26 cited by
MONet: Unsupervised Scene Decomposition and Representation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
MONet: Unsupervised Scene Decomposition and Representation
read the original abstract
The ability to decompose scenes in terms of abstract building blocks is crucial for general intelligence. Where those basic building blocks share meaningful properties, interactions and other regularities across scenes, such decompositions can simplify reasoning and facilitate imagination of novel scenarios. In particular, representing perceptual observations in terms of entities should improve data efficiency and transfer performance on a wide range of tasks. Thus we need models capable of discovering useful decompositions of scenes by identifying units with such regularities and representing them in a common format. To address this problem, we have developed the Multi-Object Network (MONet). In this model, a VAE is trained end-to-end together with a recurrent attention network -- in a purely unsupervised manner -- to provide attention masks around, and reconstructions of, regions of images. We show that this model is capable of learning to decompose and represent challenging 3D scenes into semantically meaningful components, such as objects and background elements.
Forward citations
Cited by 26 Pith papers
-
CLEVRER: CoLlision Events for Video REpresentation and Reasoning
CLEVRER introduces a diagnostic dataset for evaluating video models on causal reasoning via descriptive, explanatory, predictive, and counterfactual questions about object collision events.
-
Content-Style Identification via Differential Independence
Introduces CSDI as a structural condition for identifiability of content and style in nonlinear generative mixtures, operationalized via blockwise Jacobian orthogonality and a stochastic regularizer.
-
What-Where Transformer: A Slot-Centric Visual Backbone for Concurrent Representation and Localization
The What-Where Transformer achieves explicit what-where separation in a ViT-style backbone via concurrent token and attention-map streams, yielding emergent object discovery from attention maps and better weakly-super...
-
OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation
OA-WAM uses persistent address vectors and dynamic content vectors in object slots to enable addressable world-action prediction, improving robustness on manipulation benchmarks under scene changes.
-
Learning to Theorize the World from Observation
NEO is a probabilistic neural model that induces compositional programs as a learned Language of Thought from non-textual observations and executes them via a shared transition model to enable explanation-driven gener...
-
Mask World Model: Predicting What Matters for Robust Robot Policy Learning
Mask World Model predicts semantic mask dynamics with video diffusion and integrates it with a diffusion policy head, outperforming RGB world models on LIBERO and RLBench while showing better real-world generalization...
-
TrajTok: Learning Trajectory Tokens enables better Video Understanding
TrajTok learns adaptive trajectory tokens for videos through a unified end-to-end segmenter, improving understanding performance and efficiency over patch-based or external-pipeline tokenizers.
-
ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation
ReKep encodes robotic tasks as optimizable Python functions over 3D keypoints that are generated automatically from language and RGB-D input, enabling real-time hierarchical planning on single- and dual-arm platforms ...
-
Rethinking Object-Centric Representations for Video Dynamics Modeling
STAITUS disentangles appearance from pose in video slots, enforces spatial separation and appearance-only temporal alignment, and adds adaptive gating to improve segmentation and identity persistence.
-
TSA: Temporal Slot Activation for Persistent Object-Centric Video Representation
TSA adds learned activation scores to control slot state preservation and decoder participation in recurrent video object-centric models, improving decomposition and identity preservation on occluded videos.
-
Dual-State Slot Attention: Decoupling Appearance and Identity for Video Object-Centric Learning
DSSA decouples per-frame appearance from temporal identity in slot attention mechanisms to reduce slot swapping and improve temporal consistency in video object segmentation.
-
Slots, Transitions, Loops: Learning Composable World Models for ARC
Loop-OWM uses color-prototype slots, demonstration-conditioned task summaries, and looped transitions to model ARC rules as visual-symbolic state changes and outperforms baselines on ARC-1 and ARC-2.
-
Formalizing the Binding Problem
Introduces an information-theoretic formalization of the binding problem and a probing method to quantify binding information in deep learning model representations, tested on ViTs across challenging datasets.
-
Learning Bilevel Policies over Symbolic World Models for Long-Horizon Planning
BISON learns bilevel policies over symbolic world models to generalize long-horizon robotic planning beyond VLA and end-to-end baselines while remaining efficient even at 10,000-object scale.
-
Slot-MPC: Goal-Conditioned Model Predictive Control with Object-Centric Representations
Slot-MPC learns slot representations to build a differentiable object-centric dynamics model that supports efficient gradient-based MPC for robotic manipulation in novel situations.
-
Anatomy-Slot: Unsupervised Anatomical Factorization for Homologous Bilateral Reasoning in Retinal Diagnosis
Anatomy-Slot decomposes patch tokens into emergent anatomical slots and aligns them across eyes via bidirectional cross-attention, reporting a 4.2 AUC gain over ViT-L on ODIR-5K.
-
Anatomy-Slot: Unsupervised Anatomical Factorization for Homologous Bilateral Reasoning in Retinal Diagnosis
Anatomy-Slot improves AUC by 4.2% on ODIR-5K over a ViT-L baseline by unsupervised slot factorization and cross-eye alignment.
-
Learning to Theorize the World from Observation
NEO induces compositional latent programs as world theories from observations and executes them to enable explanation-driven generalization.
-
When Attention Collapses: Residual Evidence Modeling for Compositional Inference
Standard attention collapses on additively mixed signals because it is memoryless with respect to explained evidence, but adding multiplicative depletion with an attention bias prevents collapse and enables multi-sour...
-
Deep sprite-based image models: An analysis
A deep sprite-based image decomposition method matches SOTA unsupervised class-aware segmentation on CLEVR, scales linearly with objects, explicitly identifies categories, and fully models images interpretably.
-
3D-DLP: Self-Supervised 3D Object-Centric Scene Representation Learning
3D-DLP decomposes 3D scenes into controllable latent particles via self-supervised reconstruction for improved robotic tasks.
-
Variational Proximal Policy Optimization
VP2O maps PPO to SVGD in a MoE architecture using functional kernels and expert orthogonalization, claiming +179 ELO on Codeforces and 32% token reduction on AIME for a 33B/4B model.
-
Kuramoto Oscillatory Phase Encoding: Neuro-inspired Synchronization for Improved Learning Efficiency
KoPE adds Kuramoto-based oscillatory phase states and synchronization to Vision Transformers, improving training, parameter, and data efficiency on structured vision tasks.
-
CausalVAE as a Plug-in for World Models: Towards Reliable Counterfactual Dynamics
CausalVAE plug-in for world models preserves factual prediction and boosts counterfactual retrieval, with large gains on physics benchmarks and recovered physical interaction trends.
-
Consciousness in Artificial Intelligence: Insights from the Science of Consciousness
No current AI systems exhibit the indicator properties derived from established scientific theories of consciousness, yet there appear to be no fundamental technical obstacles to implementing those properties in futur...
-
A Compositional Framework for Open-ended Intelligence
Open-ended intelligence is formalized as the compositional closure L(P,C) of primitives P under operators C, with next primitive prediction proposed as an objective to acquire reusable primitives and grammar for lifel...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.