pith. sign in

arxiv: 2605.01799 · v2 · pith:3NX7PZZ3new · submitted 2026-05-03 · 💻 cs.CV

Embody4D: A Generalist Data Engine for Embodied 4D World Modeling

Peiyan Tu , Hanxin Zhu , Jingwen Sun , Shaojie Ren , Cong Wang , Yuyan Xu , Jiayi Luo , Xiaoqian Cheng
show 1 more author
Zhibo Chen
This is my paper

Pith reviewed 2026-07-01 00:15 UTC · model grok-4.3

classification 💻 cs.CV
keywords embodied 4D modelingnovel view synthesisvideo-to-video translationrobotic data generationspatiotemporal consistencyworld modelingmanipulation understandingview-consistent generation
0
0 comments X

The pith

Embody4D converts monocular robot videos into novel-view videos that preserve geometric and interaction consistency.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that a specialized video-to-video model can close the viewpoint gap in robot data by turning single-camera recordings into flexible multi-view 4D sequences. It does this through a synthesis pipeline that builds diverse training sets, a modulation method that maintains spatiotemporal stability during generation, and targeted attention on manipulation regions. A sympathetic reader would care because embodied agents need complete 3D spatiotemporal information for reasoning and planning, yet real-world collection of extra viewpoints is impractical. If the approach works, single-view robot datasets become sufficient to train models that generalize across camera positions. The work reports state-of-the-art visual quality and downstream gains in both simulated and physical robot tasks.

Core claim

Embody4D is a video-to-video world model that takes monocular robot videos and produces novel-view videos from arbitrary target camera poses; it achieves this by first curating a heterogeneous dataset via a 3D-aware compositional synthesis pipeline, then applying a latent confidence-aware expert modulation strategy that estimates the reliability of warped latent priors and routes regions to appropriate copy, repair, or inpaint experts, and finally using an interaction-aware attention mechanism that focuses on robotic contact regions, resulting in high-fidelity, view-consistent 4D output that improves visual benchmarks and robotic planning.

What carries the argument

The latent confidence-aware expert modulation strategy, which estimates reliability of warped latent priors and adaptively routes image regions to copy, repair, or inpaint experts.

If this is right

  • Single-view robot recordings become usable for training models that require multi-view consistency.
  • Downstream robotic planning and learning tasks receive higher-fidelity synthetic data without additional real-world camera setups.
  • Cross-embodiment generalization improves because the curated dataset mixes robotic arms with varied backgrounds.
  • Viewpoint flexibility in generated videos supports spatial reasoning that fixed-camera data cannot provide.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same pipeline could be tested on non-robotic domains such as human motion capture where multi-view data are also scarce.
  • If the expert routing proves reliable, it might reduce reliance on explicit 3D reconstruction pipelines in other video synthesis settings.
  • A direct test would measure whether policies trained on the generated multi-view data outperform those trained only on the original monocular sequences in a standardized manipulation benchmark.

Load-bearing premise

The modulation strategy can correctly judge the reliability of warped latent priors and route regions to the right expert without introducing new inconsistencies.

What would settle it

Run the model on held-out monocular robot videos, render the output from several novel viewpoints, and check whether geometric distortions or interaction artifacts appear that are absent in ground-truth multi-view captures; if such artifacts are frequent, the consistency claim fails.

Figures

Figures reproduced from arXiv: 2605.01799 by Cong Wang, Hanxin Zhu, Jiayi Luo, Jingwen Sun, Peiyan Tu, Shaojie Ren, Xiaoqian Cheng, Yuyan Xu, Zhibo Chen.

Figure 1
Figure 1. Figure 1: Introducing 4D World Model. Multiview information is crucial for embod￾ied manipulation and planning, and there is an urgent need for embodied multiview 4D world models to provide comprehensive spatial environmental representations for downstream tasks. reasoning tasks [27]. However, while the physical world is inherently three￾dimensional, most existing world models remain confined to 2D pixel space [13].… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of Embody4D. We construct paired embodied training videos in 3D via compositional synthesis and process them with a “warp-then-inpaint” ar￾chitecture. The source video is reconstructed into a point cloud and projected to the target view to produce warped RGB plus occupancy masks; these are concatenated and passed to a confidence module that adaptively injects different noise levels. Finally, a bac… view at source ↗
Figure 3
Figure 3. Figure 3: Interaction-Aware Block. This module projects Q, K, V and interaction biases via linear layers. The bias, derived from foreground masks, is injected into the guided path to prioritize manipulation regions, ensuring geometric consistency across viewpoint changes (qualitative comparison in heatmaps). where E(·) denotes a pre-trained feature encoder from the Wan model and resize(·) refers to bilinear interpol… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparisons of novel view video synthesis. view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparisons of the ablations. view at source ↗
Figure 6
Figure 6. Figure 6: Ablation on the compositional training data. view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of successful novel￾view synthesis counts under four experimen￾tal settings on seen tasks and OOD unseen tasks An embodied world model can serve as a data engine of em￾bodied training data [26, 52]. We posit the hypothesis that a 4D embodied world model can augment multi-view percep￾tion for real-world robot deploy￾ment. To substantiate this hy￾pothesis, we augment a monocu￾lar dataset using Emb… view at source ↗
Figure 8
Figure 8. Figure 8: Comparison of successful novel-view synthesis counts view at source ↗
Figure 9
Figure 9. Figure 9: Visualization results of real-world embodied experiments. view at source ↗
Figure 10
Figure 10. Figure 10: Visualization results of compositional 4D embodied data view at source ↗
Figure 11
Figure 11. Figure 11: Visualization of interaction motion priors. view at source ↗
Figure 12
Figure 12. Figure 12: Visualization results of Embody4D view at source ↗
Figure 13
Figure 13. Figure 13: Visualization results of Embody4D Source View TrajCrafter ReCamMaster EX-4D Ours view at source ↗
Figure 14
Figure 14. Figure 14: Qualitative visualization results of our method and the baselines. view at source ↗
Figure 15
Figure 15. Figure 15: Qualitative visualization results of our method and the baselines. view at source ↗
read the original abstract

Embodied agents require robust and comprehensive 3D spatiotemporal representations to support spatial reasoning, manipulation understanding, and downstream decision making. However, existing robot data are typically captured from fixed or sparse viewpoints, providing only partial and view-dependent observations, which limits multi-view perception and generalization across viewpoints. Given the difficulty of collecting additional viewpoints in real-world settings, we propose Embody4D, a dedicated video-to-video world model for embodied scenarios to bridge this observation gap by transforming a monocular robot video into novel-view videos from flexible target camera viewpoints. First, to tackle training data scarcity, we introduce a 3D-aware compositional synthesis pipeline to curate a heterogeneous dataset compositing cross-embodiment robotic arms with diverse backgrounds, promoting broad generalization. Second, to enforce geometric stability, we devise a latent confidence-aware expert modulation strategy, which estimates the reliability of warped latent priors and adaptively routes regions to copy, repair, or inpaint experts for spatiotemporally consistent 4D generation. Finally, to enhance the fidelity of the manipulation, we incorporate an interaction-aware attention mechanism that explicitly attends to the robotic interaction regions. Extensive experiments show that Embody4D achieves state-of-the-art performance on visual evaluation benchmarks, while both simulated and real-world robotic experiments further demonstrate its effectiveness as a robust data engine for synthesizing high-fidelity, view-consistent videos that empower downstream robotic planning and learning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes Embody4D, a video-to-video world model for embodied 4D world modeling that transforms monocular robot videos into novel-view videos. It introduces a 3D-aware compositional synthesis pipeline to curate a heterogeneous cross-embodiment dataset, a latent confidence-aware expert modulation strategy that estimates reliability of warped latent priors and routes regions to copy/repair/inpaint experts for spatiotemporal consistency, and an interaction-aware attention mechanism to improve manipulation fidelity. The central claims are state-of-the-art performance on visual evaluation benchmarks plus effectiveness as a data engine for downstream simulated and real-world robotic planning and learning tasks.

Significance. If the results hold, the work would be significant for embodied AI by providing a practical mechanism to overcome sparse/fixed viewpoints in robot data and generate view-consistent 4D videos at scale. The concrete routing logic in the modulation strategy and the dual evaluation on visual benchmarks plus robotic transfer are strengths that support the data-engine framing.

major comments (2)
  1. [Abstract] Abstract and Experiments: the headline SOTA claim is load-bearing for the contribution, yet the abstract supplies no metrics, baselines, or error bars; while the experiments section reportedly contains these details, the absence in the summary makes independent assessment of the central performance claim difficult from the front matter alone.
  2. [Method (latent confidence-aware expert modulation)] Latent confidence-aware expert modulation strategy: this component is load-bearing for the geometric-stability claim; the routing logic is described at a high level, but the manuscript should supply an explicit equation or pseudocode for the reliability estimation function and an ablation showing that the adaptive routing measurably improves consistency over a non-modulated baseline.
minor comments (2)
  1. [Method] The interaction-aware attention mechanism is mentioned but its precise formulation (e.g., how interaction regions are detected and how the attention mask is applied) would benefit from an equation or diagram reference.
  2. [Experiments] Ensure all visual and robotic experiment tables include standard deviations or confidence intervals so that reported gains can be evaluated for statistical reliability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback and positive assessment of the work's potential significance. We address each major comment below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [Abstract] Abstract and Experiments: the headline SOTA claim is load-bearing for the contribution, yet the abstract supplies no metrics, baselines, or error bars; while the experiments section reportedly contains these details, the absence in the summary makes independent assessment of the central performance claim difficult from the front matter alone.

    Authors: We agree that the abstract would benefit from including key quantitative results to allow readers to assess the central claims immediately. In the revised version, we will update the abstract to report specific metrics (e.g., FID, PSNR), main baselines, and a note on error bars drawn from the experiments section. revision: yes

  2. Referee: [Method (latent confidence-aware expert modulation)] Latent confidence-aware expert modulation strategy: this component is load-bearing for the geometric-stability claim; the routing logic is described at a high level, but the manuscript should supply an explicit equation or pseudocode for the reliability estimation function and an ablation showing that the adaptive routing measurably improves consistency over a non-modulated baseline.

    Authors: The current manuscript describes the routing logic and reliability estimation in Section 3.2 at a conceptual level. We acknowledge that an explicit equation and dedicated ablation would strengthen the presentation. We will add a formal equation (or pseudocode) for the reliability estimation function and include an ablation comparing the full modulated model against a non-adaptive baseline in the revised experiments section. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper describes a video-to-video world model via a 3D-aware synthesis pipeline, latent confidence-aware expert modulation, and interaction-aware attention. No equations, derivations, fitted parameters presented as predictions, or self-citation chains appear in the abstract or method summary. Claims rest on external benchmark results and robotic transfer experiments rather than internal reductions to inputs by construction. This is the expected non-finding for a descriptive systems paper without mathematical self-reference.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no identifiable free parameters, axioms, or invented entities; ledger left empty.

pith-pipeline@v0.9.1-grok · 5811 in / 1234 out tokens · 47609 ms · 2026-07-01T00:15:01.385097+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

63 extracted references · 43 canonical work pages · 14 internal anchors

  1. [1]

    GPT-4 Technical Report

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  2. [2]

    V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning

    Assran, M., Bardes, A., Fan, D., Garrido, Q., Howes, R., Muckley, M., Rizvi, A., Roberts, C., Sinha, K., Zholus, A., et al.: V-jepa 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  3. [3]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Bahmani, S., Skorokhodov, I., Rong, V., Wetzstein, G., Guibas, L., Wonka, P., Tulyakov, S., Park, J.J., Tagliasacchi, A., Lindell, D.B.: 4d-fy: Text-to-4d gener- ation using hybrid score distillation sampling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7996–8006 (2024)

    2024

  4. [4]

    Yuxuan Bian, Zhaoyang Zhang, Xuan Ju, Mingdeng Cao, Liangbin Xie, Ying Shan, and Qiang Xu

    Bai,J., Xia, M., Fu, X.,Wang, X.,Mu, L., Cao, J.,Liu, Z., Hu, H., Bai, X., Wan, P., et al.: Recammaster: Camera-controlled generative rendering from a single video. arXiv preprint arXiv:2503.11647 (2025)

    work page arXiv 2025

  5. [5]

    eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers

    Balaji, Y., Nah, S., Huang, X., Vahdat, A., Song, J., Zhang, Q., Kreis, K., Ait- tala, M., Aila, T., Laine, S., et al.: ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  6. [6]

    In: 2024 IEEE International Conference on Robotics and Automation (ICRA)

    Bharadhwaj, H., Vakil, J., Sharma, M., Gupta, A., Tulsiani, S., Kumar, V.: Roboa- gent: Generalization and efficiency in robot manipulation via semantic augmenta- tions and action chunking. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). pp. 4788–4795. IEEE (2024)

    2024

  7. [7]

    AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems

    Bu, Q., Cai, J., Chen, L., Cui, X., Ding, Y., Feng, S., Gao, S., He, X., Hu, X., Huang, X., et al.: Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.06669 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  8. [8]

    SAM 3: Segment Anything with Concepts

    Carion, N., Gustafson, L., Hu, Y.T., Debnath, S., Hu, R., Suris, D., Ryali, C., Alwala,K.V.,Khedr,H.,Huang,A.,etal.:Sam3:Segmentanythingwithconcepts. arXiv preprint arXiv:2511.16719 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  9. [9]

    Large Video Planner Enables Generalizable Robot Control

    Chen, B., Zhang, T., Geng, H., Song, K., Zhang, C., Li, P., Freeman, W.T., Malik, J., Abbeel, P., Tedrake, R., et al.: Large video planner enables generalizable robot control. arXiv preprint arXiv:2512.15840 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  10. [10]

    What is the best 3d scene representation for robotics? from geometric to foundation models,

    Deng, T., Pan, Y., Yuan, S., Li, D., Wang, C., Li, M., Chen, L., Xie, L., Wang, D., Wang, J., et al.: What is the best 3d scene representation for robotics? from geometric to foundation models. arXiv preprint arXiv:2512.03422 (2025)

    work page arXiv 2025

  11. [11]

    ACM Computing Surveys58(3), 1–38 (2025)

    Ding, J., Zhang, Y., Shang, Y., Zhang, Y., Zong, Z., Feng, J., Yuan, Y., Su, H., Li, N., Sukiennik, N., et al.: Understanding world or predicting future? a compre- hensive survey of world models. ACM Computing Surveys58(3), 1–38 (2025)

    2025

  12. [12]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Dong, Q., Fu, Y.: Memflow: Optical flow estimation and prediction with memory. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 19068–19078 (2024)

    2024

  13. [13]

    Worldscore: A unified evaluation benchmark for world generation

    Duan, H., Yu, H.X., Chen, S., Fei-Fei, L., Wu, J.: Worldscore: A unified evaluation benchmark for world generation. arXiv preprint arXiv:2504.00983 (2025) 16 F. Author et al

    work page arXiv 2025

  14. [14]

    In: Forty-first international conference on machine learning (2024)

    Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., et al.: Scaling rectified flow transformers for high-resolution image synthesis. In: Forty-first international conference on machine learning (2024)

    2024

  15. [15]

    Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot,

    Fang, H.S., Fang, H., Tang, Z., Liu, J., Wang, C., Wang, J., Zhu, H., Lu, C.: Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot. arXiv preprint arXiv:2307.00595 (2023)

    work page arXiv 2023

  16. [16]

    IEEE robotics & au- tomation magazine29(2), 46–64 (2022)

    Haddadin, S., Parusel, S., Johannsmeier, L., Golz, S., Gabl, S., Walch, F., Sabaghian, M., Jähne, C., Hausperger, L., Haddadin, S.: The franka emika robot: A reference platform for robotics research and education. IEEE robotics & au- tomation magazine29(2), 46–64 (2022)

    2022

  17. [17]

    CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers

    Hong, W., Ding, M., Zheng, W., Liu, X., Tang, J.: Cogvideo: Large-scale pretrain- ing for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  18. [18]

    arXiv preprint arXiv:2512.05672 (2025)

    Hong, Y., Lee, S., Chung, H., Ye, J.C.: Inversecrafter: Efficient video recapture as a latent domain inverse problem. arXiv preprint arXiv:2512.05672 (2025)

    work page arXiv 2025

  19. [19]

    EX-4D: Extreme viewpoint 4D video synthesis via depth watertight mesh.arXiv preprint arXiv:2506.05554, 2025

    Hu, T., Peng, H., Liu, X., Ma, Y.: Ex-4d: Extreme viewpoint 4d video synthesis via depth watertight mesh. arXiv preprint arXiv:2506.05554 (2025)

    work page arXiv 2025

  20. [20]

    Huang, Y.-W

    Huang, W., Chao, Y.W., Mousavian, A., Liu, M.Y., Fox, D., Mo, K., Fei-Fei, L.: Pointworld: Scaling 3d world models for in-the-wild robotic manipulation. arXiv preprint arXiv:2601.03782 (2026)

    work page arXiv 2026

  21. [21]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., et al.: Vbench: Comprehensive benchmark suite for video gener- ative models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21807–21818 (2024)

    2024

  22. [22]

    0.5: a vision-language-action model with open- world generalization (2025)

    Intelligence, P., Black, K., Brown, N., Darpinian, J., Dhabalia, K., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., et al.: : a vision-language-action model with open-world generalization. 0.5: a vision-language-action model with open- world generalization (2025)

    2025

  23. [23]

    In: conference on Robot Learning

    Jang, E., Irpan, A., Khansari, M., Kappler, D., Ebert, F., Lynch, C., Levine, S., Finn, C.: Bc-z: Zero-shot task generalization with robotic imitation learning. In: conference on Robot Learning. pp. 991–1002. PMLR (2022)

    2022

  24. [24]

    DreamGen: Unlocking Generalization in Robot Learning through Video World Models

    Jang,J.,Ye,S.,Lin,Z.,Xiang,J.,Bjorck,J.,Fang,Y.,Hu,F.,Huang,S.,Kundalia, K., Lin, Y.C., et al.: Dreamgen: Unlocking generalization in robot learning through video world models. arXiv preprint arXiv:2505.12705 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  25. [25]

    Reangle-a-video: 4d video generation as video-to-video translation.arXiv preprint arXiv:2503.09151, 2025

    Jeong, H., Lee, S., Ye, J.C.: Reangle-a-video: 4d video generation as video-to-video translation. arXiv preprint arXiv:2503.09151 (2025)

    work page arXiv 2025

  26. [26]

    Jiang, S

    Jiang, Y., Chen, S., Huang, S., Chen, L., Zhou, P., Liao, Y., He, X., Liu, C., Li, H., Yao, M., et al.: Enerverse-ac: Envisioning embodied environments with action condition. arXiv preprint arXiv:2505.09723 (2025)

    work page arXiv 2025

  27. [27]

    OpenVLA: An Open-Source Vision-Language-Action Model

    Kim, M.J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., et al.: Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  28. [28]

    2025.3D and 4D world modeling: a survey

    Kong, L., Yang, W., Mei, J., Liu, Y., Liang, A., Zhu, D., Lu, D., Yin, W., Hu, X., Jia, M., et al.: 3d and 4d world modeling: A survey. arXiv preprint arXiv:2509.07996 (2025)

    work page arXiv 2025

  29. [29]

    Kwon, J., Buchman, E.: Cosmos whitepaper. A Netw. Distrib. Ledgers27(1-32), 24 (2019)

    2019

  30. [30]

    arXiv preprint arXiv:2401.17807 (2024) 17

    Li, X., Zhang, Q., Kang, D., Cheng, W., Gao, Y., Zhang, J., Liang, Z., Liao, J., Cao, Y.P., Shan, Y.: Advances in 3d generation: A survey. arXiv preprint arXiv:2401.17807 (2024) 17

    work page arXiv 2024

  31. [31]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Ling, L., Sheng, Y., Tu, Z., Zhao, W., Xin, C., Wan, K., Yu, L., Guo, Q., Yu, Z., Lu, Y., et al.: Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 22160–22169 (2024)

    2024

  32. [32]

    Flow Matching for Generative Modeling

    Lipman, Y., Chen, R.T., Ben-Hamu, H., Nickel, M., Le, M.: Flow matching for generative modeling. arXiv preprint arXiv:2210.02747 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  33. [33]

    Robotransfer: Geometry-consistent video diffusion for robotic visual policy transfer.arXiv preprint arXiv:2505.23171, 2025

    Liu, L., Wang, X., Zhao, G., Li, K., Qin, W., Qiu, J., Zhu, Z., Huang, G., Su, Z.: Robotransfer: Geometry-consistent video diffusion for robotic visual policy trans- fer. arXiv preprint arXiv:2505.23171 (2025)

    work page arXiv 2025

  34. [34]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Liu, T., Huang, Z., Chen, Z., Wang, G., Hu, S., Shen, L., Sun, H., Cao, Z., Li, W., Liu, Z.: Free4d: Tuning-free 4d scene generation with spatial-temporal consistency. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 25571–25582 (2025)

    2025

  35. [35]

    Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow

    Liu, X., Gong, C., Liu, Q.: Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  36. [36]

    See4d: Pose-free 4d generation via auto-regressive video inpainting

    Lu, D., Liang, A., Huang, T., Fu, X., Zhao, Y., Ma, B., Pan, L., Yin, W., Kong, L., Ooi, W.T., et al.: See4d: Pose-free 4d generation via auto-regressive video in- painting. arXiv preprint arXiv:2510.26796 (2025)

    work page arXiv 2025

  37. [37]

    arXiv preprint arXiv:2511.19836 (2025)

    Lu, Y., Luo, W., Tu, P., Li, H., Zhu, H., Yu, Z., Wang, X., Chen, X., Peng, X., Li, X., et al.: 4dworldbench: A comprehensive evaluation framework for 3d/4d world generation models. arXiv preprint arXiv:2511.19836 (2025)

    work page arXiv 2025

  38. [38]

    Video generation models in robotics-applications, research challenges, future directions.CoRR, abs/2601.07823, 2026

    Mei, Z., Yin, T., Shorinwa, O., Badithela, A., Zheng, Z., Bruno, J., Bland, M., Zha, L., Hancock, A., Fisac, J.F., et al.: Video generation models in robotics-applications, research challenges, future directions. arXiv preprint arXiv:2601.07823 (2026)

    work page arXiv 2026

  39. [39]

    Advances in 4d generation: A survey,

    Miao, Q., Li, K., Quan, J., Min, Z., Ma, S., Xu, Y., Yang, Y., Liu, P., Luo, Y.: Advances in 4d generation: A survey. arXiv preprint arXiv:2503.14501 (2025)

    work page arXiv 2025

  40. [40]

    arXiv preprint arXiv:2507.17462 (2025)

    Nie, C., Wang, G., Lie, Z., Wang, H.: Ermv: Editing 4d robotic multi-view images to enhance embodied agents. arXiv preprint arXiv:2507.17462 (2025)

    work page arXiv 2025

  41. [41]

    Peebles,W.,Xie,S.:Scalablediffusionmodelswithtransformers.In:Proceedingsof the IEEE/CVF international conference on computer vision. pp. 4195–4205 (2023)

    2023

  42. [42]

    Shen, Y., Wei, F., Du, Z., Liang, Y., Lu, Y., Yang, J., Zheng, N., Guo, B.: Videovla:Videogeneratorscanbegeneralizablerobotmanipulators.arXivpreprint arXiv:2512.06963 (2025)

    work page arXiv 2025

  43. [43]

    Score-Based Generative Modeling through Stochastic Differential Equations

    Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., Poole, B.: Score- based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 (2020)

    work page internal anchor Pith review Pith/arXiv arXiv 2011

  44. [44]

    arXiv preprint arXiv:2411.04928 (2024)

    Sun, W., Chen, S., Liu, F., Chen, Z., Duan, Y., Zhang, J., Wang, Y.: Dimensionx: Create any 3d and 4d scenes from a single image with controllable video diffusion. arXiv preprint arXiv:2411.04928 (2024)

    work page arXiv 2024

  45. [45]

    Interndata-a1: Pioneering high-fidelity synthetic data for pre-training generalist policy.arXiv preprint arXiv:2511.16651, 2025

    Tian, Y., Yang, Y., Xie, Y., Cai, Z., Shi, X., Gao, N., Liu, H., Jiang, X., Qiu, Z., Yuan, F., et al.: Interndata-a1: Pioneering high-fidelity synthetic data for pre- training generalist policy. arXiv preprint arXiv:2511.16651 (2025)

    work page arXiv 2025

  46. [46]

    In: 2012 IEEE/RSJ international conference on intelligent robots and systems

    Todorov, E., Erez, T., Tassa, Y.: Mujoco: A physics engine for model-based control. In: 2012 IEEE/RSJ international conference on intelligent robots and systems. pp. 5026–5033. IEEE (2012)

    2012

  47. [47]

    In: 2023 IEEE 3rd International Conference on Computer Communication and Artificial Intelligence (CCAI)

    Tu, P., He, T., Yang, Z., Zhu, Z.: A mmw radar indoor mapping method based on transfer learning. In: 2023 IEEE 3rd International Conference on Computer Communication and Artificial Intelligence (CCAI). pp. 331–335. IEEE (2023) 18 F. Author et al

    2023

  48. [48]

    Advances in neural information pro- cessing systems30(2017)

    Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information pro- cessing systems30(2017)

    2017

  49. [49]

    Wan: Open and Advanced Large-Scale Video Generative Models

    Wan, T., Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.W., Chen, D., Yu, F., Zhao, H., Yang, J., et al.: Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  50. [50]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Wang, J., Chen, M., Karaev, N., Vedaldi, A., Rupprecht, C., Novotny, D.: Vggt: Visual geometry grounded transformer. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 5294–5306 (2025)

    2025

  51. [51]

    Chronosobserver: Taming 4d world with hyperspace diffusion sampling.arXiv preprint arXiv:2512.01481, 2025

    Wang, Q., Zhao, Y., Shen, P., Li, J., Li, J.: Chronosobserver: Taming 4d world with hyperspace diffusion sampling. arXiv preprint arXiv:2512.01481 (2025)

    work page arXiv 2025

  52. [52]

    EmbodiedGen: Towards a generative 3D world engine for embodied intelligence

    Wang, X., Liu, L., Cao, Y., Wu, R., Qin, W., Wang, D., Sui, W., Su, Z.: Em- bodiedgen: Towards a generative 3d world engine for embodied intelligence. arXiv preprint arXiv:2506.10600 (2025)

    work page arXiv 2025

  53. [53]

    arXiv preprint arXiv:2505.05474 (2025)

    Wen, B., Xie, H., Chen, Z., Hong, F., Liu, Z.: 3d scene generation: A survey. arXiv preprint arXiv:2505.05474 (2025)

    work page arXiv 2025

  54. [54]

    Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels

    Wu, H., Zhang, Z., Zhang, W., Chen, C., Liao, L., Li, C., Gao, Y., Wang, A., Zhang, E., Sun, W., et al.: Q-align: Teaching lmms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  55. [55]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Wu, S., Fei, H., Yang, J., Li, X., Li, J., Zhang, H., Chua, T.s.: Learning 4d panoptic scene graph generation from rich 2d visual scene. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 24539–24549 (2025)

    2025

  56. [56]

    Sv4d: Dynamic 3d content generation with multi-frame and multi-view consistency

    Xie, Y., Yao, C.H., Voleti, V., Jiang, H., Jampani, V.: Sv4d: Dynamic 3d con- tent generation with multi-frame and multi-view consistency. arXiv preprint arXiv:2407.17470 (2024)

    work page arXiv 2024

  57. [57]

    Orv: 4d occupancy-centric robot video generation.arXiv preprint arXiv:2506.03079, 2025

    Yang, X., Li, B., Xu, S., Wang, N., Ye, C., Chen, Z., Qin, M., Ding, Y., Zhu, Z., Jin, X., et al.: Orv: 4d occupancy-centric robot video generation. arXiv preprint arXiv:2506.03079 (2025)

    work page arXiv 2025

  58. [58]

    Trajectorycrafter: Redirecting camera trajectory for monocular videos via diffusion models.arXiv preprint arXiv:2503.05638, 2025

    YU, M., Hu, W., Xing, J., Shan, Y.: Trajectorycrafter: Redirecting camera trajec- tory for monocular videos via diffusion models. arXiv preprint arXiv:2503.05638 (2025)

    work page arXiv 2025

  59. [59]

    URL http://github

    Zakka, K., Tassa, Y., Contributors, M.M.: Mujoco menagerie: A collec- tion of high-quality simulation models for mujoco. URL http://github. com/deepmind/mujoco_menagerie (2022)

    2022

  60. [60]

    arXiv preprint arXiv:2512.02729 (2025)

    Zhang, Y., Gao, Z., Li, S., Chen, L.H., Liu, K., Cheng, R., Lin, X., Liu, J., Li, Z., Feng, J., et al.: Robowheel: A data engine from real-world human demonstrations for cross-embodiment robotic learning. arXiv preprint arXiv:2512.02729 (2025)

    work page arXiv 2025

  61. [61]

    Tesseract: Learning 4d embodied world models, 2025.https://arxiv.org/abs/2504.20995

    Zhen, H., Sun, Q., Zhang, H., Li, J., Zhou, S., Du, Y., Gan, C.: Tesseract: learning 4d embodied world models. arXiv preprint arXiv:2504.20995 (2025)

    work page arXiv 2025

  62. [62]

    Omniworld: A multi-domain and multi-modal dataset for 4d world modeling.arXiv preprint arXiv:2509.12201, 2025

    Zhou, Y., Wang, Y., Zhou, J., Chang, W., Guo, H., Li, Z., Ma, K., Li, X., Wang, Y., Zhu, H., et al.: Omniworld: A multi-domain and multi-modal dataset for 4d world modeling. arXiv preprint arXiv:2509.12201 (2025)

    work page arXiv 2025

  63. [63]

    robot arm and objects

    Zhu, F., Wu, H., Guo, S., Liu, Y., Cheang, C., Kong, T.: Irasim: A fine-grained world model for robot manipulation. In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision. pp. 9834–9844 (2025) 19 Appendix for Embody4D A Real-world Embodied Experiments In our real-world experiments, the input camera pose is illustrated in Fig. 9. The...

    2025