pith. sign in

arxiv: 2606.24232 · v1 · pith:N7RBU6MYnew · submitted 2026-06-23 · 💻 cs.CV · cs.GR

FiCA: Feed-forward instant Gaussian Codec Avatars from a Single Portrait Image

Kim Youwang , Zhengyu Yang , Liuhao Ge , Yu Rong , Timur Bagautdinov , Su Zhaoen , Nir Sopher , Jovan Popovi\'c
show 3 more authors
Teng Deng Tae-Hyun Oh Chen Cao
This is my paper

Pith reviewed 2026-06-26 00:39 UTC · model grok-4.3

classification 💻 cs.CV cs.GR
keywords Gaussian avatarsfeed-forward generationsingle-image avatardiffusion model3D mesh reconstructionphotorealistic renderingreal-time animationcodec avatars
0
0 comments X

The pith

A single portrait image produces a drivable photorealistic 3D Gaussian avatar in one feed-forward pass.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces FiCA, a pipeline that combines human-centric foundation models with a diffusion model to infer complete 3D head geometry and appearance from one portrait. A feed-forward refinement network then improves fidelity, after which a universal prior converts the mesh into 3D Gaussians that support real-time animation with new expressions. A sympathetic reader would care because prior avatar methods typically demand multiple views or slow per-person optimization at test time, restricting use in consumer applications. The system claims to deliver identity-preserving results that match or exceed the visual quality of slower recent approaches while remaining instantaneous.

Core claim

FiCA learns a generative mapping from partial single-portrait observations to complete and authentic 3D mesh reconstructions via a diffusion model, augments this with a feed-forward mesh refinement network that removes the need for person-specific test-time optimization, and decodes the resulting mesh through a universal prior into a set of 3D Gaussians that render as photorealistic, expression-drivable avatars.

What carries the argument

Diffusion model that maps partial visual observations to complete 3D mesh reconstruction, followed by a feed-forward refinement network and a universal prior that decodes the mesh into 3D Gaussians.

If this is right

  • Avatars faithfully represent diverse identities from single images.
  • Generated avatars surpass the visual quality of recent competing methods.
  • No person-specific test-time optimization is required.
  • Photorealistic 3D Gaussian avatars support real-time driving with novel expressions.
  • The full pipeline operates in a single feed-forward pass.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same diffusion-plus-refinement structure could be tested on single-image reconstruction of full bodies or hands.
  • Real-time performance might allow direct integration into live video or mobile AR without cloud processing.
  • If the universal Gaussian decoder generalizes across identities, it could reduce the need for large per-avatar training datasets in future work.

Load-bearing premise

The diffusion model can learn a reliable mapping from one partial portrait view to a full, identity-preserving 3D head mesh without any person-specific optimization.

What would settle it

Run the pipeline on single portraits of subjects with rare head shapes or extreme lighting, then compare the generated avatar's rendered novel views against multi-view ground-truth captures of the same person; large deviations in identity or geometry would falsify the mapping claim.

Figures

Figures reproduced from arXiv: 2606.24232 by Chen Cao, Jovan Popovi\'c, Kim Youwang, Liuhao Ge, Nir Sopher, Su Zhaoen, Tae-Hyun Oh, Teng Deng, Timur Bagautdinov, Yu Rong, Zhengyu Yang.

Figure 1
Figure 1. Figure 1: Feed-forward instant Gaussian Codec Avatars (FiCA). Our method creates drivable, photorealistic 3D Gaussian head avatars from a casually captured, single portrait image, within 5 seconds. The generated head avatars can be animated consistently across different identities in real-time, given target expressions. Please refer to the supplementary video for dynamic avatar animation results. Abstract We introdu… view at source ↗
Figure 2
Figure 2. Figure 2: FiCA: Pipeline Overview. FiCA generates a high-quality drivable Gaussian Codec Avatar from only a single portrait image, without offline face tracking or person-specific fine-tuning. We introduce three main modules: 1) UV texture and geometry diffusion model, 2) feed-forward UV refinement network, and 3) universal prior model. FiCA first employs fine-tuned Sapiens [32] models to obtain per-pixel UV and ver… view at source ↗
Figure 3
Figure 3. Figure 3: Effect of Feed-forward UV Refinement. Our UV refinement network uses rich image features from (a) input image and (b) rendering of diffusion generated mesh to refine the mesh texture and geometry, resulting in enhanced avatar fidelity and ID preservation (d). For error images, the gray area means zero error. Note that the target task of our diffusion model is differ￾ent from that of diffusion-based inpaint… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative Results. We show the animated results of our generated 3D Gaussian avatars for test IDs and novel expressions. Our FiCA generates authentic, ID-preserving avatars for diverse attributes, e.g., races, genders, ages, hairstyles, and expressions, only from a single image. Also, the input image’s visual details, such as tattoos or accessories, are faithfully reflected in the 3D Gaussian avatars. No… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative Comparison: Static Avatar. PanoHead takes ∼80 secs. to generate an avatar with per-subject GAN inver￾sion. For FiCA (ours), we visualize the textured meshes, which takes ∼5 seconds to generate. FiCA shows better completeness, especially for extreme viewpoints. Note that the FiCA meshes are later decoded into animatable 3D Gaussians with visual details. 4.3. Comparison with Competing Methods Com… view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative Comparison: Animated Avatar. We compare FiCA with recent 3D portrait animation methods [9, 10, 13, 14, 62]. Given an input portrait image of held-out identity, we generate avatars using all methods and drive them using tracked driving expression codes of the same identity. FiCA shows better avatar rendering quality, especially for extreme expressions and skin tones. We compare the quality of Fi… view at source ↗
Figure 8
Figure 8. Figure 8: Application: Feed-forward Avatar Editing. We show￾case an application scenario of FiCA. Given an input portrait image, we can use a 2D image editing method to edit images in 2D. Our feed-forward pipeline creates stylized and drivable Gaussian avatars without heuristic 3D space optimization or manipulation. model, trained to generate complete texture and geometry only from a partial UV RGB texture map. As p… view at source ↗
read the original abstract

We introduce FiCA, a Feed-forward, instant Gaussian Codec Avatar generation pipeline that creates lifelike avatars from a single portrait image. Generating a photorealistic and drivable avatar from just a single image is significantly challenging due to the limited visual information available to accurately infer the 3D appearance and geometry of human heads. To address this, we develop a novel system that combines human-centric vision foundation models with a diffusion model. This system is designed to fully exploit partial visual observations to generate lifelike human avatars. Our proposed diffusion model learns a generative mapping from these partial observations to complete and authentic 3D mesh reconstruction. Additionally, we introduce a feed-forward mesh refinement network that enhances the fidelity and identity preservation of the generated avatars, eliminating the need for person-specific test-time optimization. By leveraging a universal prior model that decodes a generated mesh into a set of 3D Gaussians, we generate a photorealistic 3D Gaussian avatar, capable of being driven with novel expressions in real-time. Our experiments demonstrate that the avatars generated by our feed-forward approach faithfully represent diverse identities and surpass the visual quality of avatars produced by recent competing methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces FiCA, a feed-forward pipeline for generating photorealistic, drivable 3D Gaussian avatars from a single portrait image. It combines human-centric vision foundation models with a diffusion model that learns a generative mapping from partial observations to complete 3D mesh reconstructions, followed by a feed-forward mesh refinement network and a universal prior decoder to produce 3D Gaussians. The system claims to eliminate person-specific test-time optimization while producing avatars that faithfully represent diverse identities, support novel expressions in real time, and surpass the visual quality of recent competing methods.

Significance. If the central claims hold with rigorous validation, the work would be significant for enabling instant, optimization-free avatar creation suitable for real-time applications in AR/VR and animation, addressing a key bottleneck in single-image 3D head reconstruction.

major comments (2)
  1. [Experiments] Experiments section: the central claim that the feed-forward approach 'surpass[es] the visual quality of avatars produced by recent competing methods' and 'faithfully represent[s] diverse identities' rests on unspecified experiments; no quantitative metrics (e.g., identity similarity scores, pose generalization error), dataset splits, ablation studies, or error bars are described to substantiate generalization of the diffusion model across pose, ethnicity, or extreme viewpoints.
  2. [Method] Method section (diffusion model description): the assertion that the diffusion model 'learns a generative mapping from these partial observations to complete and authentic 3D mesh reconstruction' is load-bearing for the no-optimization claim, yet the manuscript provides no verification (e.g., held-out extreme-pose or cross-ethnicity quantitative results) that the learned prior produces consistent back/side geometry rather than frontal-biased hallucination.
minor comments (2)
  1. [Abstract] Abstract and introduction: the phrase 'universal prior model that decodes a generated mesh into a set of 3D Gaussians' would benefit from a brief citation or reference to the specific prior work being reused.
  2. [Figures] Figure captions and results: visual comparisons would be clearer if they explicitly labeled the input portrait, generated mesh, and final Gaussian rendering for each competing method.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thorough review and for highlighting the need for stronger quantitative support of our central claims. We agree that the current manuscript would benefit from expanded experimental details and additional verification results. We address each major comment below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [Experiments] Experiments section: the central claim that the feed-forward approach 'surpass[es] the visual quality of avatars produced by recent competing methods' and 'faithfully represent[s] diverse identities' rests on unspecified experiments; no quantitative metrics (e.g., identity similarity scores, pose generalization error), dataset splits, ablation studies, or error bars are described to substantiate generalization of the diffusion model across pose, ethnicity, or extreme viewpoints.

    Authors: We acknowledge that the submitted manuscript presents the experimental claims primarily through qualitative comparisons and does not include the requested quantitative metrics, dataset splits, ablations, or error bars. This is a valid observation. In the revised version we will add: (1) identity similarity scores using a standard face recognition model such as ArcFace, (2) pose generalization error measured on held-out extreme yaw/pitch angles, (3) explicit train/test splits and cross-ethnicity evaluation, (4) ablation studies isolating the diffusion prior and mesh refinement, and (5) error bars across multiple random seeds. These additions will directly substantiate the generalization claims. revision: yes

  2. Referee: [Method] Method section (diffusion model description): the assertion that the diffusion model 'learns a generative mapping from these partial observations to complete and authentic 3D mesh reconstruction' is load-bearing for the no-optimization claim, yet the manuscript provides no verification (e.g., held-out extreme-pose or cross-ethnicity quantitative results) that the learned prior produces consistent back/side geometry rather than frontal-biased hallucination.

    Authors: We agree that quantitative verification of non-frontal geometry consistency is essential to support the claim that the diffusion model produces authentic 3D reconstructions without frontal bias. The current manuscript relies on qualitative examples for this aspect. In the revision we will report quantitative metrics (e.g., surface reconstruction error on back/side regions) on held-out extreme-pose and cross-ethnicity test sets to demonstrate that the learned prior generalizes beyond frontal observations. revision: yes

Circularity Check

0 steps flagged

No circularity: learned generative mapping with no equations or self-referential derivations.

full rationale

The paper presents FiCA as a learned pipeline: a diffusion model that learns a generative mapping from single-portrait inputs to 3D meshes, plus a feed-forward refinement network and a universal prior for Gaussian decoding. No mathematical derivations, fitted parameters renamed as predictions, or load-bearing self-citations appear in the abstract or described claims. The central result is explicitly an empirical outcome of training on data rather than any reduction of outputs to inputs by construction, satisfying the default expectation of a self-contained learned model.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review yields no explicit free parameters, axioms, or invented entities; the diffusion model and universal prior are treated as black-box learned components whose training assumptions are not stated.

pith-pipeline@v0.9.1-grok · 5773 in / 1153 out tokens · 14192 ms · 2026-06-26T00:39:03.456487+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

75 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    Ogras, and Linjie Luo

    Sizhe An, Hongyi Xu, Yichun Shi, Guoxian Song, Umit Y . Ogras, and Linjie Luo. Panohead: Geometry-aware 3d full- head synthesis in 360deg. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2, 3, 7

    2023

  2. [2]

    Bridging the gap: Studio-like avatar creation from a monocular phone capture

    ShahRukh Athar, Shunsuke Saito, Zhengyu Yang, Stanislav Pidhorsky, and Chen Cao. Bridging the gap: Studio-like avatar creation from a monocular phone capture. InEuropean Conference on Computer Vision (ECCV), 2024. 2, 4

    2024

  3. [3]

    Ffhq-uv: Normalized facial uv-texture dataset for 3d face reconstruction

    Haoran Bai, Di Kang, Haoxian Zhang, Jinshan Pan, and Linchao Bao. Ffhq-uv: Normalized facial uv-texture dataset for 3d face reconstruction. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2

    2023

  4. [4]

    Universal facial encoding of codec avatars from vr headsets.ACM Transactions on Graphics (SIGGRAPH), 43(4), 2024

    Shaojie Bai, Te-Li Wang, Chenghui Li, Akshay Venkatesh, Tomas Simon, Chen Cao, Gabriel Schwartz, Jason Saragih, Yaser Sheikh, and Shih-En Wei. Universal facial encoding of codec avatars from vr headsets.ACM Transactions on Graphics (SIGGRAPH), 43(4), 2024. 7

    2024

  5. [5]

    Black, and Victoria Fernandez Abrevaya

    Shrisha Bharadwaj, Yufeng Zheng, Otmar Hilliges, Michael J. Black, and Victoria Fernandez Abrevaya. Flare: Fast learning of animatable and relightable mesh avatars.ACM Transac- tions on Graphics (SIGGRAPH), 42:15, 2023. 2

    2023

  6. [6]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. In- structpix2pix: Learning to follow image editing instructions. InIEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), pages 18392–18402, 2023. 8

    2023

  7. [7]

    Marcel C. Buehler, Gengyan Li, Erroll Wood, Leonhard Helminger, Xu Chen, Tanmay Shah, Daoye Wang, Stephan Garbin, Sergio Orts-Escolano, Otmar Hilliges, Dmitry Lagun, J´er´emy Riviere, Paulo Gotardo, Thabo Beeler, Abhimitra Meka, and Kripasindhu Sarkar. Cafca: High-quality novel view synthesis of expressive faces from casual few-shot cap- tures.ACM Transac...

  8. [8]

    Authentic volumetric avatars from a phone scan.ACM Transactions on Graphics (SIGGRAPH), 41(4),

    Chen Cao, Tomas Simon, Jin Kyu Kim, Gabe Schwartz, Michael Zollhoefer, Shun-Suke Saito, Stephen Lombardi, Shih-En Wei, Danielle Belko, Shoou-I Yu, Yaser Sheikh, and Jason Saragih. Authentic volumetric avatars from a phone scan.ACM Transactions on Graphics (SIGGRAPH), 41(4),

  9. [9]

    Generalizable and ani- matable gaussian head avatar

    Xuangeng Chu and Tatsuya Harada. Generalizable and ani- matable gaussian head avatar. InAdvances in Neural Infor- mation Processing Systems (NeurIPS), 2024. 3, 7, 12

    2024

  10. [10]

    GPAvatar: Generaliz- able and precise head avatar from image(s)

    Xuangeng Chu, Yu Li, Ailing Zeng, Tianyu Yang, Lijian Lin, Yunfei Liu, and Tatsuya Harada. GPAvatar: Generaliz- able and precise head avatar from image(s). InInternational Conference on Learning Representations (ICLR), 2024. 3, 7

    2024

  11. [11]

    Black, and Timo Bolkart

    Radek Danecek, Michael J. Black, and Timo Bolkart. EMOCA: Emotion driven monocular face capture and anima- tion. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 5

    2022

  12. [12]

    Arcface: Additive angular margin loss for deep face recogni- tion

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recogni- tion. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 7

    2019

  13. [13]

    Portrait4d: Learning one-shot 4d head avatar synthesis using synthetic data

    Yu Deng, Duomin Wang, Xiaohang Ren, Xingyu Chen, and Baoyuan Wang. Portrait4d: Learning one-shot 4d head avatar synthesis using synthetic data. InIEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), 2024. 3, 7, 12

    2024

  14. [14]

    Portrait4d-v2: Pseudo multi-view data creates better 4d head synthesizer

    Yu Deng, Duomin Wang, and Baoyuan Wang. Portrait4d-v2: Pseudo multi-view data creates better 4d head synthesizer. In European Conference on Computer Vision (ECCV), 2024. 3, 7, 12

    2024

  15. [15]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim En- tezari, Jonas M¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. InInter- national Conference on Machine Learning (ICML), 2024. 13

    2024

  16. [16]

    Black, and Victoria Abrevaya

    Haiwen Feng, Timo Bolkart, Joachim Tesch, Michael J. Black, and Victoria Abrevaya. Towards racially unbiased skin tone estimation via scene disambiguation. InEuropean Conference on Computer Vision (ECCV), 2022. 3

    2022

  17. [17]

    Black, and Timo Bolkart

    Yao Feng, Haiwen Feng, Michael J. Black, and Timo Bolkart. Learning an animatable detailed 3D face model from in-the- wild images.ACM Transactions on Graphics (SIGGRAPH), 40(8), 2021. 3

    2021

  18. [18]

    Dynamic neural radiance fields for monocular 4d facial avatar reconstruction

    Guy Gafni, Justus Thies, Michael Zollhofer, and Matthias Niessner. Dynamic neural radiance fields for monocular 4d facial avatar reconstruction. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 2

    2021

  19. [19]

    Monocular dynamic view synthesis: A reality check

    Hang Gao, Ruilong Li, Shubham Tulsiani, Bryan Russell, and Angjoo Kanazawa. Monocular dynamic view synthesis: A reality check. InAdvances in Neural Information Processing Systems (NeurIPS), 2022. 2

    2022

  20. [20]

    Ganfit: Generative adversarial network fitting for high fidelity 3d face reconstruction

    Baris Gecer, Stylianos Ploumpis, Irene Kotsia, and Stefanos Zafeiriou. Ganfit: Generative adversarial network fitting for high fidelity 3d face reconstruction. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 2

    2019

  21. [21]

    Fast-ganfit: Generative adversarial network for high fidelity 3d face reconstruction.IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021

    Baris Gecer, Stylianos Ploumpis, Irene Kotsia, and Stefanos P Zafeiriou. Fast-ganfit: Generative adversarial network for high fidelity 3d face reconstruction.IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021. 2

    2021

  22. [22]

    Mononphm: Dynamic head reconstruction from monocular videos

    Simon Giebenhain, Tobias Kirschstein, Markos Georgopou- los, Martin R ¨unz, Lourdes Agapito, and Matthias Nießner. Mononphm: Dynamic head reconstruction from monocular videos. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2

    2024

  23. [23]

    Npga: Neural parametric gaussian avatars

    Simon Giebenhain, Tobias Kirschstein, Martin R¨unz, Lour- des Agapito, and Matthias Nießner. Npga: Neural parametric gaussian avatars. InACM Transactions on Graphics (SIG- GRAPH Asia), 2024. 2

    2024

  24. [24]

    Neural head avatars from monocular rgb videos

    Philip-William Grassal, Malte Prinzler, Titus Leistner, Carsten Rother, Matthias Nießner, and Justus Thies. Neural head avatars from monocular rgb videos. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 2

    2022

  25. [25]

    Dai, and Quoc V

    David Ha, Andrew M. Dai, and Quoc V . Le. Hypernetworks. InInternational Conference on Learning Representations (ICLR), 2017. 5

    2017

  26. [26]

    Id-sculpt: Id-aware 3d head generation from single in- the-wild portrait image

    Jinkun Hao, Junshu Tang, Jiangning Zhang, Ran Yi, Yijia Hong, Moran Li, Weijian Cao, Yating Wang, and Lizhuang 9 Ma. Id-sculpt: Id-aware 3d head generation from single in- the-wild portrait image. InAAAI Conference on Artificial Intelligence (AAAI), 2024. 2, 3, 4

    2024

  27. [27]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 5, 13

    2022

  28. [28]

    Diffrelight: Diffusion-based facial performance relighting

    Mingming He, Pascal Clausen, Ahmet Levent Tas ¸el, Li Ma, Oliver Pilarski, Wenqi Xian, Laszlo Rikker, Xueming Yu, Ryan Burgert, Ning Yu, and Paul Debevec. Diffrelight: Diffusion-based facial performance relighting. InACM Trans- actions on Graphics (SIGGRAPH Asia), New York, NY , USA,

  29. [29]

    Association for Computing Machinery. 2

  30. [30]

    Panoptic studio: A massively multiview system for social motion capture

    Hanbyul Joo, Hao Liu, Lei Tan, Lin Gui, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, and Yaser Sheikh. Panoptic studio: A massively multiview system for social motion capture. InIEEE International Conference on Computer Vision (ICCV), 2015. 2

    2015

  31. [31]

    Pippo: High-resolution multi- view humans from a single image

    Yash Kant, Ethan Weber, Jin Kyu Kim, Rawal Khirodkar, Su Zhaoen, Julieta Martinez, Igor Gilitschenski, Shunsuke Saito, and Timur Bagautdinov. Pippo: High-resolution multi- view humans from a single image. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 4, 13, 14

    2025

  32. [32]

    3d gaussian splatting for real-time radi- ance field rendering.ACM Transactions on Graphics (SIG- GRAPH), 42(4), 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk¨uhler, and George Drettakis. 3d gaussian splatting for real-time radi- ance field rendering.ACM Transactions on Graphics (SIG- GRAPH), 42(4), 2023. 1, 2, 3, 5

    2023

  33. [33]

    Sapiens: Foundation for human vision mod- els

    Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision mod- els. InEuropean Conference on Computer Vision (ECCV),

  34. [34]

    Nersemble: Multi-view radi- ance field reconstruction of human heads.ACM Transactions on Graphics (SIGGRAPH), 42(4), 2023

    Tobias Kirschstein, Shenhan Qian, Simon Giebenhain, Tim Walter, and Matthias Nießner. Nersemble: Multi-view radi- ance field reconstruction of human heads.ACM Transactions on Graphics (SIGGRAPH), 42(4), 2023. 2, 12

    2023

  35. [35]

    Fitme: Deep photorealistic 3d morphable model avatars

    Alexandros Lattas, Stylianos Moschoglou, Stylianos Ploumpis, Baris Gecer, Jiankang Deng, and Stefanos Zafeiriou. Fitme: Deep photorealistic 3d morphable model avatars. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2

    2023

  36. [36]

    Megane: Morphable eyeglass and avatar network

    Junxuan Li, Shunsuke Saito, Tomas Simon, Stephen Lom- bardi, Hongdong Li, and Jason Saragih. Megane: Morphable eyeglass and avatar network. InIEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), 2023. 8

    2023

  37. [37]

    Uravatar: Universal relightable gaussian codec avatars

    Junxuan Li, Chen Cao, Gabriel Schwartz, Rawal Khirodkar, Christian Richardt, Tomas Simon, Yaser Sheikh, and Shun- suke Saito. Uravatar: Universal relightable gaussian codec avatars. InACM Transactions on Graphics (SIGGRAPH Asia), 2024. 2, 4, 5, 14

    2024

  38. [38]

    Tianye Li, Timo Bolkart, Michael. J. Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4D scans.ACM Transactions on Graphics (SIGGRAPH Asia), 36(6), 2017. 7

    2017

  39. [39]

    Robust high-resolution video matting with tem- poral guidance

    Shanchuan Lin, Linjie Yang, Imran Saleemi, and Soumyadip Sengupta. Robust high-resolution video matting with tem- poral guidance. InIEEE Winter Conf. on Applications of Computer Vision (WACV), 2022. 5

    2022

  40. [40]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximil- ian Nickel, and Matthew Le. Flow matching for generative modeling. InInternational Conference on Learning Repre- sentations (ICLR), 2023. 4

    2023

  41. [41]

    Deep appearance models for face rendering.ACM Transactions on Graphics (SIGGRAPH), 37(4):68:1–68:13,

    Stephen Lombardi, Jason Saragih, Tomas Simon, and Yaser Sheikh. Deep appearance models for face rendering.ACM Transactions on Graphics (SIGGRAPH), 37(4):68:1–68:13,

  42. [42]

    Mixture of volumetric primitives for efficient neural rendering.ACM Transactions on Graphics (SIGGRAPH), 40(4), 2021

    Stephen Lombardi, Tomas Simon, Gabriel Schwartz, Michael Zollhoefer, Yaser Sheikh, and Jason Saragih. Mixture of volumetric primitives for efficient neural rendering.ACM Transactions on Graphics (SIGGRAPH), 40(4), 2021. 1, 2, 4

    2021

  43. [43]

    Wonder3d: Single im- age to 3d using cross-domain diffusion

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Single im- age to 3d using cross-domain diffusion. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 4

    2024

  44. [44]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. InIEEE Con- ference on Computer Vision and Pattern Recognition (CVPR),

  45. [45]

    Facelift: Single image to 3d head with view generation and gs-lrm.arXiv preprint, 2412.17812, 2024

    Weijie Lyu, Yi Zhou, Ming-Hsuan Yang, and Zhixin Shu. Facelift: Single image to 3d head with view generation and gs-lrm.arXiv preprint, 2412.17812, 2024. 3

    work page arXiv 2024

  46. [46]

    Pixel codec avatars

    Shugao Ma, Tomas Simon, Jason Saragih, Dawei Wang, Yuecheng Li, Fernando De La Torre, and Yaser Sheikh. Pixel codec avatars. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 1

    2021

  47. [47]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. InEuropean Conference on Computer Vision (ECCV),

  48. [48]

    From audio to photoreal embodiment: Synthesizing humans in conversations

    Evonne Ng, Javier Romero, Timur Bagautdinov, Shaojie Bai, Trevor Darrell, Angjoo Kanazawa, and Alexander Richard. From audio to photoreal embodiment: Synthesizing humans in conversations. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 7

    2024

  49. [49]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InIEEE International Conference on Computer Vision (ICCV), 2023. 4, 13

    2023

  50. [50]

    Rasterized edge gra- dients: Handling discontinuities differentiably

    Stanislav Pidhorskyi, Tomas Simon, Gabriel Schwartz, He Wen, Yaser Sheikh, and Jason Saragih. Rasterized edge gra- dients: Handling discontinuities differentiably. InEuropean Conference on Computer Vision (ECCV), 2024. 5

    2024

  51. [51]

    SDXL: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M¨uller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. InInternational Conference on Learning Representations (ICLR), 2024. 4, 13

    2024

  52. [52]

    Barron, and Ben Mildenhall

    Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. InInternational Conference on Learning Representations (ICLR), 2022. 3 10

    2022

  53. [53]

    Avatar fingerprinting for authorized use of synthetic talking-head videos

    Ekta Prashnani, Koki Nagano, Shalini De Mello, David Lue- bke, and Orazio Gallo. Avatar fingerprinting for authorized use of synthetic talking-head videos. InEuropean Conference on Computer Vision (ECCV), 2024. 14

    2024

  54. [54]

    Gaussiana- vatars: Photorealistic head avatars with rigged 3d gaussians

    Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, and Matthias Nießner. Gaussiana- vatars: Photorealistic head avatars with rigged 3d gaussians. InIEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), 2024. 7

    2024

  55. [55]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InInternational Conference on Machine Learning (ICML), 2021. 3, 4, 8

    2021

  56. [56]

    Filntisis, Radek Danecek, Vic- toria F

    George Retsinas, Panagiotis P. Filntisis, Radek Danecek, Vic- toria F. Abrevaya, Anastasios Roussos, Timo Bolkart, and Petros Maragos. 3d facial expressions through analysis-by- neural-synthesis. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 5

    2024

  57. [57]

    Bermano, and Daniel Cohen-Or

    Daniel Roich, Ron Mokady, Amit H. Bermano, and Daniel Cohen-Or. Pivotal tuning for latent-based editing of real images.ACM Transactions on Graphics (SIGGRAPH), 42(1),

  58. [58]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 4

    2022

  59. [59]

    FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces

    Andreas R¨ossler, Davide Cozzolino, Luisa Verdoliva, Chris- tian Riess, Justus Thies, and Matthias Nießner. Faceforensics: A large-scale video dataset for forgery detection in human faces.arXiv preprint, 1803.09179, 2018. 14

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  60. [60]

    Relightable gaussian codec avatars

    Shunsuke Saito, Gabriel Schwartz, Tomas Simon, Junxuan Li, and Giljoo Nam. Relightable gaussian codec avatars. In IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), 2024. 1, 4

    2024

  61. [61]

    Hyperextended lightface: A facial attribute analysis framework

    Sefik Ilkin Serengil and Alper Ozpinar. Hyperextended lightface: A facial attribute analysis framework. InInter- national Conference on Engineering and Emerging Technolo- gies (ICEET), 2021. 7

    2021

  62. [62]

    V oodoo xp: Expressive one-shot head reenactment for vr telepresence

    Phong Tran, Egor Zakharov, Long-Nhat Ho, Liwen Hu, Adil- bek Karmanov, Aviral Agarwal, McLean Goldwhite, Ari- ana Bermudez Venegas, Anh Tuan Tran, and Hao Li. V oodoo xp: Expressive one-shot head reenactment for vr telepresence. ACM Transactions on Graphics (SIGGRAPH Asia), 2024. 3

    2024

  63. [63]

    V oodoo 3d: V olumetric portrait disen- tanglement for one-shot 3d head reenactment

    Phong Tran, Egor Zakharov, Long-Nhat Ho, Anh Tuan Tran, Liwen Hu, and Hao Li. V oodoo 3d: V olumetric portrait disen- tanglement for one-shot 3d head reenactment. InIEEE Con- ference on Computer Vision and Pattern Recognition (CVPR),

  64. [64]

    Diffusers: State-of-the-art diffusion models

    Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-art diffusion models. https://github.com/huggingface/diffusers,

  65. [65]

    Flashavatar: High-fidelity head avatar with efficient gaus- sian embedding

    Jun Xiang, Xuan Gao, Yudong Guo, and Juyong Zhang. Flashavatar: High-fidelity head avatar with efficient gaus- sian embedding. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2

    2024

  66. [66]

    Demystifying CLIP data

    Hu Xu, Saining Xie, Xiaoqing Tan, Po-Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. Demystifying CLIP data. InInternational Conference on Learning Repre- sentations (ICLR), 2024. 4

    2024

  67. [67]

    Humbi: A large multiview dataset of human body expressions and benchmark challenge.IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(1):623–640,

    Jae Shin Yoon, Zhixuan Yu, Jaesik Park, and Hyun Soo Park. Humbi: A large multiview dataset of human body expressions and benchmark challenge.IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(1):623–640,

  68. [68]

    A large-scale 3d face mesh video dataset via neural re-parameterized optimization.Trans- actions on Machine Learning Research (TMLR), 2024

    Kim Youwang, Lee Hyun, Kim Sung-Bin, Suekyeong Nam, Janghoon Ju, and Tae-Hyun Oh. A large-scale 3d face mesh video dataset via neural re-parameterized optimization.Trans- actions on Machine Learning Research (TMLR), 2024. 7

    2024

  69. [69]

    ELITE: Efficient Gaussian Head Avatar from a Monocular Video via Learned Initialization and TEst- time Generative Adaptation

    Kim Youwang, Lee Hyoseok, Park Subin, Gerard Pons-Moll, and Tae-Hyun Oh. ELITE: Efficient Gaussian Head Avatar from a Monocular Video via Learned Initialization and TEst- time Generative Adaptation. InCVPR, 2026. 2

    2026

  70. [70]

    Humbi: A large multiview dataset of human body expressions

    Zhixuan Yu, Jae Shin Yoon, In Kyu Lee, Prashanth Venkatesh, Jaesik Park, Jihun Yu, and Hyun Soo Park. Humbi: A large multiview dataset of human body expressions. InIEEE Con- ference on Computer Vision and Pattern Recognition (CVPR),

  71. [71]

    Magicbrush: A manually annotated dataset for instruction- guided image editing

    Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction- guided image editing. InAdvances in Neural Information Processing Systems (NeurIPS), 2023. 8

    2023

  72. [72]

    Instant volumetric head avatars

    Wojciech Zielonka, Timo Bolkart, and Justus Thies. Instant volumetric head avatars. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2 11 FiCA: Feed-forward Instant Gaussian Codec Avatars from a Single Portrait Image — Supplementary Material — Kim Youwang1,2∗ Zhengyu Yang1 Liuhao Ge1 Yu Rong1 Timur Bagautdinov1 Su Zhaoen1 Nir Sop...

    2023

  73. [73]

    Diffusion-based Avatar Texture and Geometry Generation from a Single Image

    Feed-forward Gaussian Codec Avatar Generation from a Single Portrait Image 3 3.1. Diffusion-based Avatar Texture and Geometry Generation from a Single Image . . . . . . 3 3.2. Feed-forward UV Refinement Network . . . 4 3.3. Decoding Mesh into Drivable Gaussian Codec Avatar via Universal Prior Model . . . . . . 5

  74. [74]

    Datasets

    Experiments 5 4.1. Datasets . . . . . . . . . . . . . . . . . . . . 5 4.2. Qualitative Results . . . . . . . . . . . . . . 6 4.3. Comparison with Competing Methods . . . . 6 4.4. Ablation Study . . . . . . . . . . . . . . . . 8

  75. [75]

    Video for Summary & Visual Results 12 B

    Conclusion, Discussion and Limitations 8 A. Video for Summary & Visual Results 12 B. More Results 12 C. Details of FiCA Pipeline 13 C.1. Fine-tuned Sapiens for UV , Normal and Ver- tex Coordinates Prediction . . . . . . . . . 13 C.2. Latent Diffusion Model . . . . . . . . . . . 13 C.3. Feed-forward UV Refinement Net . . . . . . 14 C.4. Universal Prior Mod...