REVIEW 11 cited by
DreamDiffusion: Generating High-Quality Images from Brain EEG Signals
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
DreamDiffusion: Generating High-Quality Images from Brain EEG Signals
read the original abstract
This paper introduces DreamDiffusion, a novel method for generating high-quality images directly from brain electroencephalogram (EEG) signals, without the need to translate thoughts into text. DreamDiffusion leverages pre-trained text-to-image models and employs temporal masked signal modeling to pre-train the EEG encoder for effective and robust EEG representations. Additionally, the method further leverages the CLIP image encoder to provide extra supervision to better align EEG, text, and image embeddings with limited EEG-image pairs. Overall, the proposed method overcomes the challenges of using EEG signals for image generation, such as noise, limited information, and individual differences, and achieves promising results. Quantitative and qualitative results demonstrate the effectiveness of the proposed method as a significant step towards portable and low-cost ``thoughts-to-image'', with potential applications in neuroscience and computer vision. The code is available here \url{https://github.com/bbaaii/DreamDiffusion}.
Forward citations
Cited by 11 Pith papers
-
Let EEG Models Learn EEG
JET is a conditional flow matching framework that generates EEG as continuous raw sequences with added constraints for spectral and temporal properties, achieving over 40% lower TS-FID than prior discrete denoising me...
-
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation
Janus decouples visual encoding into task-specific pathways inside a single autoregressive transformer to unify multimodal understanding and generation while outperforming earlier unified models.
-
When VR Meets BCI: (Un)Observable Brainwave-aware Privacy Reconstruction in the Metaverse via Unrestricted Inbuilt Motion Sensors
BraVeSpy reconstructs brain EEG signals from VR headset motion sensors to infer unobservable perceptions and sensitive activities, achieving 52-67% accuracy on images and over 85% on activity fingerprinting.
-
Visualizing the Invisible: Generative Visual Grounding Empowers Universal EEG Understanding in MLLMs
Generative Visual Grounding creates visual proxy images from EEG to enhance MLLM understanding of brain signals beyond text-only alignment.
-
Visualizing the Invisible: Generative Visual Grounding Empowers Universal EEG Understanding in MLLMs
Generative Visual Grounding creates instance-specific visual proxy images from EEG signals to enhance MLLM understanding of brain activity beyond text-only alignment.
-
Bridging Brain and Semantics: A Hierarchical Framework for Semantically Enhanced fMRI-to-Video Reconstruction
CineNeuron improves fMRI-to-video reconstruction by combining bottom-up semantic enrichment with top-down Mixture-of-Memories integration and outperforms prior methods on benchmarks.
-
What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment
A new multiview EEG representation framework achieves state-of-the-art 54.8% Top-1 accuracy in within-subject zero-shot visual classification on THINGS-EEG.
-
Mapping Whisper Representations to Human ECoG Responses with Interpretable Time-Resolved Neural Encoding
The paper introduces a time-resolved neural encoder combining Whisper embeddings with recurrent temporal modeling and soft attention to predict ECoG responses, finding strongest alignment in intermediate layers and an...
-
EEG2Vision: A Multimodal EEG-Based Framework for 2D Visual Reconstruction in Cognitive Neuroscience
EEG2Vision reconstructs images from EEG using diffusion models plus LLM-guided boosting, with reconstruction quality holding up reasonably as electrode count drops from 128 to 24 channels.
-
BRAIN: Bias-Mitigation Continual Learning Approach to Vision-Brain Understanding
BRAIN uses bias-mitigation continual learning with a new de-bias contrastive loss and angular forgetting mitigation to achieve SOTA performance on vision-brain understanding benchmarks despite brain signal inconsisten...
-
Synthetic Data Generation for Brain-Computer Interfaces: Overview, Benchmarking, and Future Directions
A survey that taxonomizes synthetic brain signal generation methods into four categories, benchmarks them on motor imagery, seizure detection, SSVEP, and auditory attention tasks, and outlines evaluation principles an...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.