Make-A-Video: Text-to-Video Generation without Text-Video Data
Pith reviewed 2026-05-11 01:08 UTC · model grok-4.3
The pith
A method turns text into videos by extending image generators with motion learned separately from unlabeled footage.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Make-A-Video decomposes the temporal U-Net and attention tensors into separate spatial and temporal approximations and then runs a spatial-temporal pipeline that includes a video decoder, an interpolation model, and two super-resolution models. The system re-uses a pre-trained text-to-image model for visual content and text alignment while adding motion learned from unsupervised video. The outcome is state-of-the-art text-to-video output in resolution, frame rate, text faithfulness, and overall quality, achieved without any paired text-video training data.
What carries the argument
Spatial-temporal decomposition of U-Net and attention tensors together with a multi-stage pipeline of video decoder, interpolation, and super-resolution models.
If this is right
- Text-to-video training becomes faster because visual and language representations are reused rather than learned from scratch.
- Paired text-video datasets are no longer required to reach competitive performance.
- The generated videos carry over the aesthetic variety and fantastical content already present in current text-to-image systems.
- High-resolution and high-frame-rate results are produced by chaining the dedicated interpolation and super-resolution stages.
Where Pith is reading between the lines
- The same separation of appearance learning from motion learning could be tried on other data-scarce generation tasks such as 3D or audio synthesis.
- Modular pipelines like this one may reduce the total compute needed when extending image models to new domains.
- The approach opens a route to video editing or animation tools that start from a single text prompt and then refine motion independently.
Load-bearing premise
Motion patterns taken from unlabeled video can be added to a text-to-image model through these modules without creating visible motion artifacts or weakening how well the output matches the original text prompt.
What would settle it
A side-by-side evaluation on the same text prompts where Make-A-Video outputs show more flickering, unnatural object trajectories, or lower text-video alignment scores than models trained directly on paired text-video data.
read the original abstract
We propose Make-A-Video -- an approach for directly translating the tremendous recent progress in Text-to-Image (T2I) generation to Text-to-Video (T2V). Our intuition is simple: learn what the world looks like and how it is described from paired text-image data, and learn how the world moves from unsupervised video footage. Make-A-Video has three advantages: (1) it accelerates training of the T2V model (it does not need to learn visual and multimodal representations from scratch), (2) it does not require paired text-video data, and (3) the generated videos inherit the vastness (diversity in aesthetic, fantastical depictions, etc.) of today's image generation models. We design a simple yet effective way to build on T2I models with novel and effective spatial-temporal modules. First, we decompose the full temporal U-Net and attention tensors and approximate them in space and time. Second, we design a spatial temporal pipeline to generate high resolution and frame rate videos with a video decoder, interpolation model and two super resolution models that can enable various applications besides T2V. In all aspects, spatial and temporal resolution, faithfulness to text, and quality, Make-A-Video sets the new state-of-the-art in text-to-video generation, as determined by both qualitative and quantitative measures.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Make-A-Video, a text-to-video generation method that transfers progress from text-to-image (T2I) models by learning appearance and text alignment from paired text-image data while acquiring motion dynamics from unsupervised video footage. It introduces a spatial-temporal decomposition of the U-Net and attention tensors, combined with a multi-stage pipeline (video decoder, temporal interpolation, and super-resolution models) to produce high-resolution, high-frame-rate videos without requiring paired text-video data. The central claim is that this yields state-of-the-art results in spatial/temporal resolution, text faithfulness, and perceptual quality, as measured by both qualitative examples and quantitative metrics.
Significance. If the quantitative claims hold, the work is significant because it demonstrates a practical route to high-quality T2V generation that sidesteps the scarcity of paired text-video data, accelerates training by reusing T2I representations, and inherits the diversity of modern image generators. The decomposition approach and modular pipeline are reusable for other video synthesis tasks and could reduce compute barriers in the field.
major comments (2)
- [§4] §4 (Experiments): The SOTA claim is central but rests on quantitative comparisons whose details (specific metrics such as FVD, CLIP similarity, or human preference scores, exact baselines, and effect sizes) are not summarized in the abstract and must be verified against prior T2V methods; without these numbers and ablations on the spatial-temporal modules, the superiority cannot be assessed.
- [§3.2] §3.2 (Spatial-Temporal Decomposition): The approximation of full temporal U-Net and attention tensors in space and time is described at a high level; the paper must supply the precise tensor factorization or insertion points (e.g., which layers receive the temporal attention) to confirm that motion transfer occurs without degrading text conditioning or introducing systematic artifacts.
minor comments (2)
- [Abstract] The abstract and introduction could more explicitly list the quantitative metrics and baselines used to support the SOTA statement.
- [Figures] Figure captions for qualitative results should include the exact text prompts and frame counts to aid reproducibility.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback on our manuscript. We address each major comment below with clarifications from the paper and propose targeted revisions to strengthen the presentation of our results and technical details.
read point-by-point responses
-
Referee: [§4] §4 (Experiments): The SOTA claim is central but rests on quantitative comparisons whose details (specific metrics such as FVD, CLIP similarity, or human preference scores, exact baselines, and effect sizes) are not summarized in the abstract and must be verified against prior T2V methods; without these numbers and ablations on the spatial-temporal modules, the superiority cannot be assessed.
Authors: We agree that a concise summary of the key quantitative results would improve accessibility. Section 4 reports FVD, CLIP similarity, and human preference scores against baselines including CogVideo and other recent T2V methods, with effect sizes and ablations on the spatial-temporal modules detailed in Tables 1-3 and Section 4.3 (plus appendix). The abstract states the SOTA outcome but does not list the numbers. We will revise the abstract to include a brief summary of the primary metrics and baselines while retaining the existing detailed comparisons in the experiments section. revision: partial
-
Referee: [§3.2] §3.2 (Spatial-Temporal Decomposition): The approximation of full temporal U-Net and attention tensors in space and time is described at a high level; the paper must supply the precise tensor factorization or insertion points (e.g., which layers receive the temporal attention) to confirm that motion transfer occurs without degrading text conditioning or introducing systematic artifacts.
Authors: We appreciate this request for greater precision. Section 3.2 describes the decomposition of the U-Net and attention tensors into separate spatial and temporal factors, with temporal attention inserted after spatial attention in the decoder blocks to enable motion modeling while preserving the pretrained text-image conditioning pathway. To address the comment directly, we will add a detailed diagram and explicit layer specifications (including tensor shapes and insertion points) in the revised Section 3.2. revision: yes
Circularity Check
No significant circularity in derivation chain
full rationale
The paper presents Make-A-Video as a pipeline that inherits appearance from external pretrained T2I models and motion from separate unsupervised video data. It describes a spatial-temporal decomposition of U-Net/attention tensors plus a multi-stage generation pipeline (video decoder, interpolation, super-resolution). No load-bearing step reduces by construction to a self-fit, self-definition, or self-citation chain; the central claim is a concrete engineering combination of independent pretrained components rather than a tautological prediction. The SOTA assertion rests on external qualitative/quantitative evaluation, not internal re-derivation of inputs.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Decomposing full temporal U-Net and attention tensors into separate spatial and temporal approximations preserves sufficient modeling capacity for coherent video generation.
Forward citations
Cited by 60 Pith papers
-
PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generation
PhyAVBench supplies the first benchmark and contrastive metric that measures whether text-to-audio-video models respect real-world audio physics across controlled prompt pairs.
-
OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers
OrbitQuant is a data-agnostic PTQ technique for DiTs that uses RPBH rotation in a normalized basis to enable a single codebook across all inputs, achieving SOTA low-bit performance on FLUX.1, CogVideoX and similar models.
-
HCMS: Head-Chunked Multi-Stream Pipeline for Communication-Computation Overlap in Long-Sequence Parallel Attention
HCMS partitions multi-head attention into chunks and pipelines them across dual CUDA streams to overlap communication and computation, delivering 10-17.5% speedup over Ulysses for 31K-56K token sequences.
-
Goku: A Million-Scale Universal Dataset and Benchmark for Instruction-Based Video Editing
Goku provides a 2M-pair dataset for multi-task structural video editing, Goku-Edit model with MLLM and dual-branch design, and Goku-Bench yielding up to 8% gains in instruction following.
-
Goku: A Million-Scale Universal Dataset and Benchmark for Instruction-Based Video Editing
Goku supplies a 2M-scale dataset, synthesis pipeline, decoupled dual-branch model, and 1000-case benchmark for multi-task instruction-based video editing, reporting up to 8% gains in instruction following.
-
MemoBench: Benchmarking World Modeling in Dynamically Changing Environments
MemoBench curates 360 ground-truth clips and an evaluation suite to diagnose memory consistency failures in video models when objects change state while out of view.
-
MemoBench: Benchmarking World Modeling in Dynamically Changing Environments
MemoBench is a new diagnostic benchmark with automated and VQA metrics that evaluates memory consistency in video models under disappear-and-reappear in dynamic environments.
-
MemoBench: Benchmarking World Modeling in Dynamically Changing Environments
MemoBench is a new diagnostic benchmark with 360 synthetic and real clips plus VQA evaluation that tests memory consistency in video models under the disappear-and-reappear paradigm in dynamically changing environments.
-
Net-Ev$^2$: A Generative Simulator for Network Event Evolution
Net-Ev² proposes a two-stage generative simulator with structure-guided masked pre-training and topology-aware diffusion using graph U-Net down/upsampling to model network event evolution from text inputs, plus a new ...
-
FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusion
FadeMem introduces distance-aware KV memory consolidation for autoregressive video diffusion that builds a temporal hierarchy with power-law merging to preserve short-term dynamics and long-range coherence under fixed...
-
Dream.exe: Can Video Generation Models Dream Executable Robot Manipulation?
Dream.exe evaluates 8 video generation models on 101 manipulation tasks by converting generated videos into executable robot trajectories in a simulator, finding measurable success rates that visual metrics do not predict.
-
SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generation
SafeGen-Bench is a benchmark with 10 malicious categories that evaluates conditional T2V models on paired start frames and text prompts, finding unsafety scores up to 44.5 and 80% guardrail failure rate.
-
MBench: A Comprehensive Benchmark on Memory Capability for Video World Models
MBench is a new benchmark that quantifies long-term memory in video world models via three hierarchical consistency dimensions evaluated on curated real videos.
-
YoCausal: How Far is Video Generation from World Model? A Causality Perspective
YoCausal benchmark shows video diffusion models detect the arrow of time but lack genuine causal understanding relative to humans.
-
CRONOS: Benchmarking Counterfactual Physical Consistency in Video Models
CRONOS benchmark shows recent open-source video generators fail to preserve physical consistency under controlled changes to viewpoint, scene, object category, and appearance.
-
MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation
MSAVBench is the first comprehensive benchmark for multi-shot audio-video generation featuring four dimensions, challenging scenarios, and an adaptive hybrid evaluation framework that achieves 91.5% Spearman correlati...
-
MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation
MSAVBench is the first comprehensive benchmark for multi-shot audio-video generation, spanning video, audio, shot, and reference dimensions with an adaptive evaluation framework that reaches 91.5% Spearman correlation...
-
Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controls
Aero-World adapts a pretrained latent diffusion transformer for action-conditioned aerial video generation by injecting inertial action tokens and using a frozen latent-space Physics Probe for inertial consistency sup...
-
Functionalization via Structure Completion and Motion Rectification
Object functionalization is cast as neural graph completion over a functional graph of parts, contacts, and motions, followed by geometry realization that also rectifies erroneous motions, demonstrated on furniture wi...
-
StreamingEffect: Real-Time Human-Centric Video Effect Generation
StreamingEffect enables real-time 720p human-centric video effect generation on one GPU via teacher-student distillation, keyframe control, and a new 130K video dataset.
-
TeDiO: Temporal Diagonal Optimization for Training-Free Coherent Video Diffusion
TeDiO regularizes temporal diagonals in diffusion transformer attention maps to produce smoother video motion while keeping per-frame quality intact.
-
R-DMesh: Video-Guided 3D Animation via Rectified Dynamic Mesh Flow
R-DMesh generates high-fidelity 4D meshes aligned to video by disentangling base mesh, motion, and a learned rectification jump offset inside a VAE, then using Triflow Attention and rectified-flow diffusion.
-
GTA: Advancing Image-to-3D World Generation via Geometry Then Appearance Video Diffusion
GTA generates 3D worlds from single images via a two-stage video diffusion process that prioritizes geometry before appearance to improve structural consistency.
-
Beyond Text Prompts: Visual-to-Visual Generation as A Unified Paradigm
Proposes V2V-Zero, a training-free framework replacing text conditioning with VLM final-layer hidden states from visual pages, achieving 0.85 on GenEval and 32.7/100 on new Simple-V2V Bench across models including vid...
-
Curated Synthetic Data Doesn't Have to Collapse: A Theoretical Study of Generative Retraining with Pluralistic Preferences
Recursive generative retraining with pluralistic preferences converges to a stable diverse distribution that satisfies a weighted Nash bargaining solution.
-
DCR: Counterfactual Attractor Guidance for Rare Compositional Generation
DCR uses a counterfactual attractor and projection-based repulsion to suppress default completion bias in diffusion models, improving fidelity for rare compositional prompts while preserving quality.
-
Structured Diffusion Bridges: Inductive Bias for Denoising Diffusion Bridges
Structured diffusion bridges with alignment constraints achieve near fully-paired quality in modality translation while working effectively in unpaired and semi-paired regimes.
-
TrajShield: Trajectory-Level Safety Mediation for Defending Text-to-Video Models Against Jailbreak Attacks
TrajShield is a training-free defense that reduces jailbreak success rates by 52.44% on average in text-to-video models by localizing and neutralizing risks through trajectory simulation and causal intervention.
-
Cutscene Agent: An LLM Agent Framework for Automated 3D Cutscene Generation
Cutscene Agent uses a multi-agent LLM system and a new toolkit for game engine control to automate end-to-end 3D cutscene generation, evaluated on the introduced CutsceneBench.
-
$Z^2$-Sampling: Zero-Cost Zigzag Trajectories for Semantic Alignment in Diffusion Models
Z²-Sampling implicitly realizes zero-cost zigzag trajectories for curvature-aware semantic alignment in diffusion models by reducing multi-step paths via operator dualities and temporal caching while synthesizing a di...
-
Sparse Forcing: Native Trainable Sparse Attention for Real-time Autoregressive Diffusion Video Generation
Sparse Forcing adds a native trainable sparsity mechanism and PBSA kernel to autoregressive diffusion video models, yielding higher VBench scores and 1.1-1.27x speedups on 5s to 1min generations.
-
ViVa: A Video-Generative Value Model for Robot Reinforcement Learning
ViVa turns a video generator into a value model for robot RL that jointly forecasts future states and task value, yielding better performance on real-world box assembly when integrated with RECAP.
-
HumANDiff: Articulated Noise Diffusion for Motion-Consistent Human Video Generation
HumANDiff improves motion consistency in human video generation by sampling diffusion noise on an articulated human body template and adding joint appearance-motion prediction plus a geometric consistency loss.
-
Training-Free Refinement of Flow Matching with Divergence-based Sampling
Flow Divergence Sampler refines flow matching by computing velocity field divergence to correct ambiguous intermediate states during inference, improving fidelity in text-to-image and inverse problem tasks.
-
Omni-NegCLIP: Enhancing CLIP with Front-Layer Contrastive Fine-Tuning for Comprehensive Negation Understanding
Omni-NegCLIP improves CLIP's negation understanding by up to 52.65% on presence-based and 12.50% on absence-based tasks through front-layer fine-tuning with specialized contrastive losses.
-
ChopGrad: Pixel-Wise Losses for Latent Video Diffusion via Truncated Backpropagation
ChopGrad truncates backpropagation to local frame windows in video diffusion models, reducing memory from linear in frame count to constant while enabling pixel-wise loss fine-tuning.
-
FrameDiT: Diffusion Transformer with Matrix Attention for Efficient Video Generation
FrameDiT proposes Matrix Attention for DiTs to achieve SOTA video generation with improved temporal coherence and efficiency comparable to local factorized attention.
-
PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generation
PhyAVBench provides the first systematic benchmark and metric for audio-physics grounding in T2AV, I2AV, and V2A models using controlled prompt pairs and real video ground truth.
-
One-to-All Animation: Alignment-Free Character Animation and Image Pose Transfer
One-to-All Animation enables alignment-free character animation and image pose transfer via self-supervised outpainting reformulation, reference extraction, hybrid fusion attention, identity-robust pose control, and t...
-
ASTRA: Let Arbitrary Subjects Transform in Video Editing
ASTRA is a plug-and-play training-free method for precise multi-subject video editing that uses prompt-guided multimodal alignment and prior-based mask retargeting to avoid attention dilution and boundary issues.
-
History-Guided Video Diffusion
DFoT enables flexible history conditioning in video diffusion, with history guidance methods that boost temporal consistency and support long rollouts.
-
Self-Correcting Text-to-Video Generation with Misalignment Detection and Localized Refinement
VideoRepair detects text-video misalignments via MLLM-generated questions and performs localized, region-preserving refinement to improve alignment in existing T2V diffusion models.
-
RoboDreamer: Learning Compositional World Models for Robot Imagination
RoboDreamer factorizes video generation using language primitives to achieve compositional generalization in robot world models, outperforming monolithic baselines on unseen goals in RT-X.
-
Learning Interactive Real-World Simulators
UniSim learns a universal real-world simulator from orchestrated diverse datasets, enabling zero-shot deployment of policies trained purely in simulation.
-
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
A new shared video-image tokenizer enables large language models to surpass diffusion models on standard visual generation benchmarks.
-
AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning
A single motion module trained on videos adds temporally coherent animation to any personalized text-to-image model derived from the same base without additional tuning.
-
Generative Semantic Communication: Diffusion Models Beyond Bit Recovery
A generative semantic communication system that sends compressed semantic information and uses diffusion models with spatially-adaptive normalizations to reconstruct high-quality, semantically consistent images even u...
-
Imagen Video: High Definition Video Generation with Diffusion Models
Imagen Video generates high-definition text-conditional videos via a cascade of base and super-resolution diffusion models, achieving high fidelity and controllability.
-
EMOSH: Expressive Motion and Shape Disentanglement for Human Animation
EMOSH proposes an Expressive Human Model with disentangled parameters, coarse-to-fine motion injection, and spatially-aligned conditioning to generate high-fidelity expressive human videos without driving-subject shap...
-
MemoBench: Benchmarking World Modeling in Dynamically Changing Environments
MemoBench curates 360 clips and an evaluation suite to test video models on recovering updated object states after disappear-and-reappear in changing environments.
-
LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing
LiveEdit distills a bidirectional video foundation model into a unidirectional streaming editor via three-stage training plus mask caching to reach 12.66 FPS with stable edits.
-
OmniDirector: General Multi-Shot Camera Cloning without Cross-Paired Data
OmniDirector introduces a grid-based camera representation and hierarchical prompt agent for multi-shot camera cloning in video diffusion models trained on million-scale unpaired data.
-
Streaming Video Generation with Streaming Force Control
StreamForce presents a unified causal model for force-controllable streaming video generation using a new force representation and distillation pipeline, claiming SOTA force adherence and 16.6 FPS performance.
-
DSA: Dynamic Step Allocation for Fast Autoregressive Video Generation
DSA adds a jointly trained confidence head to autoregressive video diffusion models that dynamically allocates fewer or more denoising steps per frame, achieving 22.63 FPS real-time generation on H100 while matching V...
-
Lumos-Nexus: Efficient Frequency Bridging with Homogeneous Latent Space for Video Unified Models
Lumos-Nexus is a training-efficient video generation framework using two-stage alignment of a lightweight model followed by progressive frequency bridging to a high-fidelity generator in homogeneous latent space, plus...
-
DRFusion: Drift-Resilient Temporally Consistent Infrared-Visible Video Fusion
DRFusion uses Stabilized History Guidance, Soft Temporal Anchoring, and Decoupled Structure-Motion Adaptation to achieve drift-resilient temporal consistency in infrared-visible video fusion.
-
Head Forcing: Long Autoregressive Video Generation via Head Heterogeneity
Head Forcing assigns tailored KV cache strategies to local, anchor, and memory attention heads plus head-wise RoPE re-encoding to extend autoregressive video generation from seconds to minutes without training.
-
R-DMesh: Video-Guided 3D Animation via Rectified Dynamic Mesh Flow
R-DMesh proposes a VAE-based disentanglement of base mesh, motion trajectories, and rectification offset plus Triflow Attention and rectified-flow diffusion to produce 4D meshes aligned to video despite initial pose mismatch.
-
Beyond Text Prompts: Visual-to-Visual Generation as A Unified Paradigm
V2V-Zero adapts frozen VLMs for visual conditioning via hidden states from specification pages, scoring 0.85 on GenEval and 32.7 on a new seven-task benchmark while revealing capability hierarchies in attribute bindin...
-
FIS-DiT: Breaking the Few-Step Video Inference Barrier via Training-Free Frame Interleaved Sparsity
FIS-DiT achieves 2.11-2.41x speedup on video DiT models in few-step regimes with negligible quality loss by exploiting frame-wise sparsity and consistency through a training-free interleaved execution strategy.
Reference graph
Works this paper leans on
-
[2]
Language Models are Few-Shot Learners
URL https://arxiv.org/abs/2005.14165. Franc ¸ois Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1251–1258,
work page internal anchor Pith review Pith/arXiv arXiv 2005
-
[3]
arXiv preprint arXiv:2204.14217 , eprint =
Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang. Cogview2: Faster and better text-to-image generation via hierarchical transformers. arXiv preprint arXiv:2204.14217,
-
[4]
Make-a-scene: Scene-based text-to-image generation with human priors.ArXiv, abs/2203.13131, 2022
URLhttps://arxiv. org/abs/2203.13131. Songwei Ge, Thomas Hayes, Harry Yang, Xi Yin, Guan Pang, David Jacobs, Jia-Bin Huang, and Devi Parikh. Long video generation with time-agnostic vqgan and time-sensitive transformer. ECCV,
-
[5]
In: IEEE/CVF Conference on Computer Vision and Pat- tern Recognition Workshops (CVPRW)
doi: 10.1109/CVPRW50498.2020.00193. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. NIPS,
-
[6]
Denoising Diffusion Probabilistic Models
URL https://arxiv.org/abs/2006.11239. Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models,
work page internal anchor Pith review arXiv 2006
-
[7]
URL https://arxiv.org/abs/2204.03458. Seunghoon Hong, Dingdong Yang, Jongwook Choi, and Honglak Lee. Inferring semantic layout for hierarchical text-to-image synthesis. In CVPR, pp. 7986–7994,
work page internal anchor Pith review arXiv
-
[8]
CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers
URL https://arxiv.org/ abs/2205.15868. Yitong Li, Martin Min, Dinghan Shen, David Carlson, and Lawrence Carin. Video generation from text. In AAAI, volume 32,
work page internal anchor Pith review arXiv
-
[9]
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692, 2019a. URL http://arxiv.org/abs/1907.11692. Yue Liu, Xin Wang, Yitian Yuan, and Wenwu Zhu. Cross-modal dual learning for sentence-to- video ge...
work page internal anchor Pith review Pith/arXiv arXiv 1907
-
[10]
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741,
work page internal anchor Pith review arXiv
-
[11]
Hierarchical Text-Conditional Image Generation with CLIP Latents
URL https://arxiv.org/abs/ 2204.06125. Fitsum Reda, Janne Kontkanen, Eric Tabellion, Deqing Sun, Caroline Pantofaru, and Brian Curless. Film: Frame interpolation for large motion. arXiv preprint arXiv:2202.04901,
work page internal anchor Pith review Pith/arXiv arXiv
-
[12]
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
URL https://arxiv.org/abs/ 2205.11487. Masaki Saito, Shunta Saito, Masanori Koyama, and Sosuke Kobayashi. Train sparsely, generate densely: Memory-efficient unsupervised training of high-resolution temporal gan. International Journal of Computer Vision, 128(10):2586–2606,
work page internal anchor Pith review arXiv
-
[13]
UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402,
work page internal anchor Pith review arXiv
-
[14]
URL https://arxiv. org/abs/1706.03762. Chenfei Wu, Lun Huang, Qianxi Zhang, Binyang Li, Lei Ji, Fan Yang, Guillermo Sapiro, and Nan Duan. Godiva: Generating open-domain videos from natural descriptions. arXiv preprint arXiv:2104.14806, 2021a. Chenfei Wu, Jian Liang, Lei Ji, Fan Yang, Yuejian Fang, Daxin Jiang, and Nan Duan. N¨Uwa: Visual synthesis pre-tra...
work page internal anchor Pith review Pith/arXiv arXiv
-
[15]
Vector-quantized Image Modeling with Improved VQGAN
Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved vqgan. arXiv preprint arXiv:2110.04627,
work page internal anchor Pith review arXiv
-
[16]
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
12 Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. Scaling autoregressive models for content-rich text-to-image generation, 2022a. URL https://arxiv.org/abs/2206.10789. Sihy...
work page internal anchor Pith review arXiv
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.