pith. sign in

arxiv: 2302.12288 · v1 · pith:M4ZW2724new · submitted 2023-02-23 · 💻 cs.CV

ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth

Shariq Farooq Bhat , Reiner Birkl , Diana Wofk , Peter Wonka , Matthias M\"uller This is my paper

Pith reviewed 2026-05-14 22:09 UTC · model grok-4.3

classification 💻 cs.CV
keywords depth estimationmonocular depthzero-shot transferrelative depthmetric depthgeneralization
0
0 comments X

The pith

A model pre-trained on relative depths from twelve datasets and fine-tuned on metric depth achieves strong zero-shot generalization while preserving scale.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper seeks to overcome the limitation that depth estimation models either generalize across scenes but ignore absolute scale or achieve metric accuracy only within the training distribution. It does this by first training on relative depth predictions from a large collection of datasets and then attaching and fine-tuning lightweight metric-specific heads. A latent classifier decides which head to use for each image at test time. If successful, this produces a single network that works on both indoor and outdoor scenes it has never seen, with metric outputs that remain consistent with physical distances.

Core claim

Pre-training a depth network on relative depth from twelve different datasets, followed by fine-tuning separate metric bins modules on NYU Depth v2 and KITTI, and routing inputs via a latent classifier, yields the first model that can train jointly on indoor and outdoor data without performance loss and generalizes metric depth to eight unseen datasets.

What carries the argument

The metric bins module, which learns to adjust the centers and widths of depth bins for each domain to produce metric-scale outputs, selected by a latent classifier that identifies the appropriate domain from image features.

If this is right

  • Without any pre-training the approach already improves state-of-the-art relative error on the NYU indoor dataset.
  • Pre-training on twelve datasets then fine-tuning on NYU improves relative absolute error by 21 percent.
  • The model can be trained jointly on NYU and KITTI with no significant performance drop.
  • Zero-shot transfer reaches eight previously unseen indoor and outdoor datasets.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The routing mechanism may allow easy addition of new domains by training only a new head and classifier update.
  • This separation of relative and metric learning could apply to other scale-sensitive tasks such as surface normal estimation or camera pose.
  • Real-world deployment in mixed environments like autonomous driving in cities with indoor navigation would benefit from the domain-agnostic routing.

Load-bearing premise

The latent classifier must correctly identify which metric head to use for each input image, even when the image comes from an unseen domain or lies between domains.

What would settle it

A drop in accuracy on the eight unseen test datasets or on images that are hard to classify as indoor or outdoor would indicate the routing step fails to preserve the claimed performance.

read the original abstract

This paper tackles the problem of depth estimation from a single image. Existing work either focuses on generalization performance disregarding metric scale, i.e. relative depth estimation, or state-of-the-art results on specific datasets, i.e. metric depth estimation. We propose the first approach that combines both worlds, leading to a model with excellent generalization performance while maintaining metric scale. Our flagship model, ZoeD-M12-NK, is pre-trained on 12 datasets using relative depth and fine-tuned on two datasets using metric depth. We use a lightweight head with a novel bin adjustment design called metric bins module for each domain. During inference, each input image is automatically routed to the appropriate head using a latent classifier. Our framework admits multiple configurations depending on the datasets used for relative depth pre-training and metric fine-tuning. Without pre-training, we can already significantly improve the state of the art (SOTA) on the NYU Depth v2 indoor dataset. Pre-training on twelve datasets and fine-tuning on the NYU Depth v2 indoor dataset, we can further improve SOTA for a total of 21% in terms of relative absolute error (REL). Finally, ZoeD-M12-NK is the first model that can jointly train on multiple datasets (NYU Depth v2 and KITTI) without a significant drop in performance and achieve unprecedented zero-shot generalization performance to eight unseen datasets from both indoor and outdoor domains. The code and pre-trained models are publicly available at https://github.com/isl-org/ZoeDepth .

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces ZoeDepth, a monocular depth estimation framework that pre-trains on 12 datasets for relative depth and fine-tunes on NYU Depth v2 and KITTI using domain-specific metric bins modules. A latent classifier routes each input image to the appropriate metric head at inference time. The flagship ZoeD-M12-NK model reports a 21% REL improvement on NYU, enables joint training on NYU+KITTI without significant performance drop, and achieves strong zero-shot generalization to eight unseen indoor and outdoor datasets.

Significance. If the empirical results hold under the routing mechanism, the work provides a practical bridge between relative-depth generalization and metric-scale accuracy, with the first demonstrated joint multi-domain metric training and broad zero-shot transfer. Public code release aids verification of the reported gains.

major comments (2)
  1. [Experiments section (zero-shot evaluation tables)] Experiments section (zero-shot evaluation tables): no confusion matrix, per-dataset routing accuracy, or forced-wrong-head ablation is reported for the latent classifier on the eight unseen datasets. Because the central claim of domain-agnostic metric performance rests on correct routing to the NYU vs. KITTI metric bins module, the absence of these diagnostics leaves open the possibility that misrouting inflates the reported REL/RMSE numbers on ambiguous inputs.
  2. [§3.3 (metric bins module and latent classifier)] §3.3 (metric bins module and latent classifier): the training objective and architecture details for the latent classifier are not fully specified (e.g., loss, number of classes, how it is trained jointly or separately). This makes it difficult to assess whether the routing is learned reliably or could be a post-hoc selection effect.
minor comments (2)
  1. [Figure 2] Figure 2 (architecture diagram): the flow from relative encoder through the latent classifier to the metric heads is not labeled with tensor dimensions or explicit routing logic, making the inference path harder to follow.
  2. [Table 1] Table 1 (NYU results): the baseline comparisons should explicitly state whether the competing methods were also pre-trained on the same 12 relative-depth datasets or only on standard supervised splits.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback and positive assessment of the significance of ZoeDepth. We address the two major comments point by point below. Both points identify areas where the manuscript can be strengthened with additional details and experiments, which we will incorporate in the revised version.

read point-by-point responses

  1. Referee: Experiments section (zero-shot evaluation tables): no confusion matrix, per-dataset routing accuracy, or forced-wrong-head ablation is reported for the latent classifier on the eight unseen datasets. Because the central claim of domain-agnostic metric performance rests on correct routing to the NYU vs. KITTI metric bins module, the absence of these diagnostics leaves open the possibility that misrouting inflates the reported REL/RMSE numbers on ambiguous inputs.

    Authors: We agree that these diagnostics are important for validating the routing mechanism. In the revised manuscript we will add: (1) a confusion matrix of routing decisions across the eight unseen datasets, (2) per-dataset routing accuracy numbers, and (3) a forced-wrong-head ablation that reports the degradation in REL/RMSE when the model is deliberately routed to the incorrect metric bins module. These additions will directly address the concern that misrouting could be inflating the zero-shot numbers and will make the evidence for correct domain-agnostic routing explicit. revision: yes

  2. Referee: §3.3 (metric bins module and latent classifier): the training objective and architecture details for the latent classifier are not fully specified (e.g., loss, number of classes, how it is trained jointly or separately). This makes it difficult to assess whether the routing is learned reliably or could be a post-hoc selection effect.

    Authors: We apologize for the incomplete specification in §3.3. The latent classifier is a lightweight two-layer MLP with two output classes (NYU vs. KITTI domain). It is trained jointly with the metric heads using cross-entropy loss on the ground-truth domain labels during the fine-tuning stage; it is not trained separately or applied post-hoc. We will expand §3.3 with the exact architecture, loss function, number of classes, and joint training procedure so that readers can fully reproduce and assess the reliability of the learned routing. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical training and routing on standard benchmarks

full rationale

The paper describes a standard supervised pipeline: pre-train a relative-depth backbone on 12 datasets, attach domain-specific metric bins modules, fine-tune on NYU and KITTI, and train a latent classifier to route inputs at inference. All performance numbers (REL, RMSE, zero-shot transfer) are obtained by direct evaluation on held-out test sets; no equation or claimed prediction is shown to equal a fitted parameter or self-citation by construction. The latent classifier is an ordinary learned component whose accuracy is measured on the same benchmarks, not presupposed by the reported metrics. The work is therefore self-contained against external data and does not reduce any central claim to its own inputs.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

This is an empirical deep-learning paper; it relies on standard assumptions that convolutional backbones learn transferable features from supervised depth labels and that a small classifier can separate indoor/outdoor domains from latent features.

free parameters (1)
  • number of bins in metric bins module
    Chosen per domain head to discretize metric depth ranges; exact count not stated in abstract but affects scale accuracy.
axioms (1)
  • domain assumption Standard supervised learning on depth labels produces generalizable features when pre-trained on diverse relative-depth datasets.
    Invoked in the pre-training stage on 12 datasets.

pith-pipeline@v0.9.0 · 5586 in / 1227 out tokens · 27773 ms · 2026-05-14T22:09:01.166711+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models

    cs.CV 2026-06 unverdicted novelty 7.0

    Introduces MultiDepth-3k benchmark revealing diverse layer preferences across depth models on ambiguous scenes, with Laplacian Visual Prompting altering outputs for some frozen models and best pair reaching 75.5% ML-SRA.

  2. Open-Vocabulary BEV Segmentation with 3D-Aware Geometric Constraints

    cs.CV 2026-06 unverdicted novelty 7.0

    OVBEVSeg enables open-vocabulary BEV segmentation via 2D-to-BEV pseudo-labeling, joint per-scene optimization, and 3D distillation, outperforming closed-set methods by 15.3 mIoU on unseen nuScenes categories while usi...

  3. DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images

    cs.CV 2026-06 unverdicted novelty 7.0

    DepthMaster unifies metric monocular depth estimation for perspective and panoramic images by patching panoramas into perspective views, adding a consistency loss and virtual cameras, and training mostly on perspectiv...

  4. Consistent Yet Wrong: Evidence Insensitivity in Spatial Vision-Language Models

    cs.CV 2026-06 unverdicted novelty 7.0

    VLMs produce highly consistent but inaccurate answers to metric spatial queries across views, indicating prior-driven collapse rather than evidence-sensitive reasoning.

  5. Honey, I Shrunk the Arc de Triomphe!

    cs.CV 2026-06 unverdicted novelty 7.0

    Introduces MetricScenes dataset with metric grounding from geo-tags and stereo, plus Poisson depth completion, showing fine-tuned MoGe-2 reduces scale-collapse in open scenes.

  6. Honey, I Shrunk the Arc de Triomphe!

    cs.CV 2026-06 unverdicted novelty 7.0

    MetricScenes dataset from web photos and stereo imagery, plus a two-stage Poisson depth completion method, allows fine-tuning MoGe-2 to mitigate scale-collapse in metric monocular geometry while preserving benchmark p...

  7. TROPHIES: Temporal Reconstruction of Places, Humans, and Cameras from Multi-view Videos

    cs.CV 2026-06 unverdicted novelty 7.0

    TROPHIES introduces a unified framework for human-scene-camera reconstruction from multi-view videos, achieving globally aligned and physically plausible 4D outputs on EgoHuman and EgoExo4D.

  8. SeeGroup: Multi-Layer Depth Estimation of Transparent Surfaces via Self-Determined Grouping

    cs.CV 2026-05 unverdicted novelty 7.0

    SeeGroup formulates per-pixel multi-layer depth as a point process with permutation-invariant likelihood to support arbitrary groupings, raising quadruplet relative depth accuracy from 61.34% to 70.09% on the LayeredD...

  9. Dual-Pathway Geometry-Aware MLLM for Spatial Intelligence

    cs.CV 2026-05 unverdicted novelty 7.0

    GAMSI is a dual-pathway Geometry-Aware MLLM using Metric-Structure Decoupled Queries and Expert-Guided Visual Grounding on RGB inputs alone, trained on a new 152k-sample MTS dataset to reach SOTA on seven spatial benchmarks.

  10. WideDepth: Millimeter-Accurate Benchmark for Fisheye Depth Estimation

    cs.CV 2026-05 unverdicted novelty 7.0

    WideDepth supplies the first millimeter-accurate indoor fisheye depth benchmark together with a stereo generation pipeline and model adaptation technique.

  11. H-Flow: Self-supervised Human Scene Flow via Physics-inspired Joint Multi-modal Learning

    cs.CV 2026-05 unverdicted novelty 7.0

    H-Flow learns dense human scene flow from monocular video via joint pose and depth prediction in a multi-head transformer, using physics-inspired geometric and biomechanical priors for self-supervision, and introduces...

  12. Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth

    cs.CV 2026-05 unverdicted novelty 7.0

    Depth2Pose is a new evaluation framework for monocular depth estimators that uses relative camera pose accuracy as a task-driven proxy and introduces the D2P dataset of challenging out-of-distribution scenes.

  13. LAMP: Localization Aware Multi-camera People Tracking in Metric 3D World

    cs.CV 2026-05 unverdicted novelty 7.0

    LAMP tracks 3D human motion from moving multi-camera headsets by converting 2D detections to a unified metric 3D world frame via device localization and fitting with an end-to-end spatio-temporal transformer.

  14. DP-SfM: Dual-Pixel Structure-from-Motion without Scale Ambiguity

    cs.CV 2026-05 unverdicted novelty 7.0

    Dual-pixel defocus blur enables absolute scale estimation in SfM without reference objects or calibration.

  15. Towards Realistic and Consistent Orbital Video Generation via 3D Foundation Priors

    cs.CV 2026-04 unverdicted novelty 7.0

    A video generation approach conditions a base model with multi-scale 3D latent features and a cross-attention adapter to produce geometrically realistic and consistent orbital videos from one image.

  16. LiftFormer: Lifting and Frame Theory Based Monocular Depth Estimation Using Depth and Edge Oriented Subspace Representation

    cs.CV 2026-04 unverdicted novelty 7.0

    LiftFormer transforms monocular depth prediction into depth-oriented geometric and edge-aware subspace representations via lifting and frame theory, achieving state-of-the-art results on standard datasets.

  17. EndoVGGT: GNN-Enhanced Depth Estimation for Surgical 3D Reconstruction

    cs.CV 2026-03 unverdicted novelty 7.0

    EndoVGGT uses a dynamic DeGAT graph attention module to improve depth estimation and non-rigid 3D reconstruction in surgery, reporting 24.6% PSNR and 9.1% SSIM gains on SCARED with zero-shot generalization to new domains.

  18. VGGT-360: Geometry-Consistent Zero-Shot Panoramic Depth Estimation

    cs.CV 2026-03 unverdicted novelty 7.0

    VGGT-360 delivers geometry-consistent zero-shot panoramic depth by converting panoramas into multi-view 3D reconstructions via VGGT models and three plug-and-play correction modules, then reprojecting the result.

  19. RAD: Retrieval-Augmented Monocular Metric Depth Estimation for Underrepresented Classes

    cs.CV 2026-02 unverdicted novelty 7.0

    RAD retrieves semantically similar RGB-D context samples for low-confidence regions and fuses them via matched cross-attention to cut relative absolute depth error by 29.2% on NYU Depth v2 underrepresented classes whi...

  20. Bridging Visual and Wireless Sensing via a Unified Radiation Field for 3D Radio Map Construction

    cs.NI 2026-01 unverdicted novelty 7.0

    URF-GS creates a single radiation field from visual and wireless observations via 3D Gaussian splatting to predict radio signals at any location and configuration with higher accuracy and fewer samples than prior NeRF...

  21. Geometry-Aware Cross Modal Alignment for Light Field-LiDAR Semantic Segmentation

    cs.CV 2025-10 unverdicted novelty 7.0

    Proposes the first light field-LiDAR semantic segmentation dataset and the Mlpfseg network, which improves mIoU by 1.71 over image-only and 2.38 over point-cloud-only baselines via feature completion and depth percept...

  22. Materialist: Physically Based Editing Using Single-Image Inverse Rendering

    cs.CV 2025-01 unverdicted novelty 7.0

    Materialist performs single-image inverse rendering via neural-initialized progressive differentiable rendering to enable physically consistent material editing, object insertion, relighting, and transparency edits wi...

  23. 3D-VLA: A 3D Vision-Language-Action Generative World Model

    cs.CV 2024-03 unverdicted novelty 7.0

    3D-VLA is a new embodied foundation model that uses a 3D LLM plus aligned diffusion models to generate future images and point clouds for improved reasoning and action planning in 3D environments.

  24. AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World

    cs.CV 2026-06 unverdicted novelty 6.0

    AerialMetric is a new benchmark dataset and evaluation suite for adapting monocular metric depth estimation models to real-world UAV aerial views.

  25. Error-Conditioned Neural Solvers

    cs.LG 2026-06 unverdicted novelty 6.0

    Error-Conditioned Neural Solvers improve PDE prediction accuracy by using the residual field as network input for learned corrections, outperforming residual-minimization methods by up to 10x on turbulent flows and ge...

  26. Beyond Aesthetics: Quantifying Information Loss in Turbid Scenes

    cs.CV 2026-06 unverdicted novelty 6.0

    Introduces the TUB dataset of 1320 real turbid underwater images and PCD metric showing strong correlation with instance segmentation performance where standard metrics fail.

  27. TaskNPoint: How to Teach Your Humanoid to Hit a Backhand in Minutes

    cs.RO 2026-06 unverdicted novelty 6.0

    TaskNPoint lets humanoid robots learn dynamic skills such as tennis backhands from single short human video demonstrations plus under one hour of single-GPU simulation training, achieving zero-shot generalization to n...

  28. ACEsplat: Accelerated 3D Gaussian Scene Regression via RGB and Poses Only

    cs.RO 2026-06 unverdicted novelty 6.0

    ACEsplat performs per-scene 3D Gaussian Splatting from RGB and poses only via a two-stage self-supervised SCR and optimization pipeline, achieving 29-33 dB PSNR on benchmarks in 15-25 minutes.

  29. Modality Forcing for Scalable Spatial Generation

    cs.CV 2026-06 unverdicted novelty 6.0

    Modality Forcing lets a single DiT produce image and depth outputs in any order after training on sparse real-world depth, with larger image-pretrained models yielding better depth accuracy and a 57% AbsRel reduction ...

  30. Consistent Yet Wrong: Evidence Insensitivity in Spatial Vision-Language Models

    cs.CV 2026-06 conditional novelty 6.0

    Leading VLMs show high cross-view consistency paired with low metric accuracy on distance queries, indicating evidence-insensitive reasoning rather than geometric grounding.

  31. Enabling Extensible Embodied Capabilities with Tools

    cs.RO 2026-05 unverdicted novelty 6.0

    Introduces Embodied Tool Protocol and tool externalization to improve embodied AI performance on perception and cognition tasks, with measured gains but limits on execution capabilities.

  32. Unified Panoramic Geometry Estimation via Multi-View Foundation Models

    cs.CV 2026-05 unverdicted novelty 6.0

    PaGeR is a framework that lifts perspective 3D foundation models to omnidirectional images through mixed training, enabling unified prediction of scale-invariant depth, metric depth, surface normals, and sky masks fro...

  33. Stabilizing Streaming Video Geometry via Dynamic Feature Normalization

    cs.CV 2026-05 unverdicted novelty 6.0

    DyFN is a lightweight recurrent module that dynamically normalizes latent feature statistics to remove scale-shift drift and achieve state-of-the-art temporal consistency in streaming monocular geometry estimation whi...

  34. UfM*: Uncertainty from Motion* for DNN Depth Estimation Using Gaussians

    cs.RO 2026-05 unverdicted novelty 6.0

    UfM* uses Gaussian mixtures to compute multiview disagreement for uncertainty in depth estimation with single inference per image, reducing energy and memory use.

  35. Resolving Representation Ambiguity in Feedforward Novel View Synthesis Transformer via Semantic-Spatial Decoupling

    cs.CV 2026-05 unverdicted novelty 6.0

    Decouples semantic and spatial tokens in NVS transformers to resolve representation ambiguity, yielding consistent gains with near-zero added latency.

  36. Unlocking Dense Metric Depth Estimation in VLMs

    cs.CV 2026-05 unverdicted novelty 6.0

    DepthVLM converts a standard VLM into a dense metric depth predictor by attaching a lightweight head and training under unified vision-text supervision, outperforming prior VLMs and some pure vision models on a new in...

  37. Unlocking Dense Metric Depth Estimation in VLMs

    cs.CV 2026-05 unverdicted novelty 6.0

    DepthVLM attaches a depth head to VLMs for native dense metric depth prediction alongside language outputs using a two-stage unified training schedule and a new indoor-outdoor benchmark.

  38. Featurising Pixels from Dynamic 3D Scenes with Linear In-Context Learners

    cs.CV 2026-04 unverdicted novelty 6.0

    LILA learns temporally consistent semantic and geometric pixel features from uncurated videos via linear in-context learning on off-the-shelf depth and motion cues, yielding empirical gains on video object segmentatio...

  39. Last-Layer-Centric Feature Recombination: Unleashing 3D Geometric Knowledge in DINOv3 for Monocular Depth Estimation

    cs.CV 2026-04 unverdicted novelty 6.0

    Layer analysis of DINOv3 shows non-uniform 3D geometric knowledge concentrated in deeper layers, enabling a last-layer-centric recombination module that improves monocular depth estimation accuracy to state-of-the-art levels.

  40. SS3D: End2End Self-Supervised 3D from Web Videos

    cs.CV 2026-04 unverdicted novelty 6.0

    SS3D pretrains an end-to-end feed-forward 3D estimator on filtered YouTube-8M videos via SfM self-supervision, MVS filtering, and expert distillation, delivering stronger zero-shot transfer and fine-tuning than prior ...

  41. SS3D: End2End Self-Supervised 3D from Web Videos

    cs.CV 2026-04 unverdicted novelty 6.0

    SS3D pretrains an end-to-end 3D estimator on filtered YouTube-8M videos via SfM self-supervision, achieving improved zero-shot transfer and fine-tuning over prior baselines.

  42. In Depth We Trust: Reliable Monocular Depth Supervision for Gaussian Splatting

    cs.CV 2026-04 unverdicted novelty 6.0

    A selective regularization framework lets scale-ambiguous monocular depth priors improve Gaussian Splatting geometry and rendering by isolating and supervising only ill-posed regions.

  43. Stepper: Stepwise Immersive Scene Generation with Multiview Panoramas

    cs.CV 2026-03 unverdicted novelty 6.0

    Stepper uses stepwise panoramic expansion with a multi-view 360-degree diffusion model and geometry reconstruction to produce high-fidelity, structurally consistent immersive 3D scenes from text.

  44. R4Det: 4D Radar-Camera Fusion for High-Performance 3D Object Detection

    cs.CV 2026-03 unverdicted novelty 6.0

    R4Det fuses 4D radar and camera inputs via panoramic depth fusion, deformable gated temporal fusion without ego pose, and instance-guided refinement to reach state-of-the-art 3D detection on TJ4DRadSet and VoD.

  45. OpenVO: Open-World Visual Odometry with Temporal Dynamics Awareness

    cs.CV 2026-02 unverdicted novelty 6.0

    OpenVO estimates ego-motion from monocular dashcam footage with varying observation rates and uncalibrated cameras by encoding temporal dynamics in a two-frame regression framework and using 3D priors from foundation ...

  46. GeCo: Evaluating Geometric Consistency for Video Generation via Motion and Structure

    cs.CV 2025-12 unverdicted novelty 6.0

    GeCo is a new geometry-based metric that produces dense maps of motion and structure inconsistencies in video generation by fusing residual motion and depth priors.

  47. Think Before You Drive: World Model-Inspired Multimodal Grounding for Autonomous Vehicles

    cs.CV 2025-12 unverdicted novelty 6.0

    ThinkDeeper introduces a world-model-based reasoning step that predicts future spatial states to improve multimodal visual grounding for autonomous vehicles, achieving top results on Talk2Car and other benchmarks.

  48. PAGE-4D: VGGT-4D Perception via Disentangled Pose and Geometry Estimation

    cs.CV 2025-10 unverdicted novelty 6.0

    PAGE-4D is a feedforward extension of VGGT that uses a dynamics-aware aggregator and mask to disentangle pose estimation from geometry reconstruction in videos with moving objects.

  49. Depth Anything V2

    cs.CV 2024-06 unverdicted novelty 6.0

    Depth Anything V2 delivers finer, more robust monocular depth predictions by replacing real labeled images with synthetic data, scaling the teacher model, and using large-scale pseudo-labeled real images for student training.

  50. Neural Surface Reconstruction from Sparse Views Using Epipolar Geometry

    cs.CV 2024-06 unverdicted novelty 6.0

    EpiS improves generalizable neural surface reconstruction from sparse views by guiding epipolar feature aggregation with cost volumes, using an epipolar transformer, and applying pretrained monocular depth constraints...

  51. SCOPE: Scale-Consistent One-Pass Estimation of 3D Geometry

    cs.CV 2026-06 unverdicted novelty 5.0

    SCOPE uses affine-invariant 3D point maps with shared parameters and three consistency innovations to estimate 3D geometry from extended monocular videos, reporting 24.2% and 34.9% error reductions on ScanNet.

  52. VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matching

    cs.CV 2026-05 unverdicted novelty 5.0

    VolFill uses a hybrid 3D VAE to compress sparse truncated unsigned distance function grids into latent space and a latent Diffusion Transformer to denoise complete scenes, conditioned on geometry foundation models, ou...

  53. PointACT: Vision-Language-Action Models with Multi-Scale Point-Action Interaction

    cs.RO 2026-05 unverdicted novelty 5.0

    PointACT proposes a 3D-aware dual-system VLA policy using multi-scale point-action interaction with bottleneck window self-attention, achieving 10% higher success rates on RLBench-10Tasks over prior pretrained VLAs.

  54. Understanding Model Behavior in Monocular Polyp Sizing

    cs.CV 2026-05 accept novelty 5.0

    Monocular polyp sizing models achieve moderate performance by exploiting examination behavior cues rather than true metric scales, with scale information and segmentation robustness acting as independent bottlenecks.

  55. Efficient 3D Content Reconstruction and Generation

    cs.CV 2026-05 unverdicted novelty 5.0

    Presents Instant3D for rapid text/image-to-3D generation via multi-view diffusion plus feed-forward reconstruction, and FastMap for 10x faster structure-from-motion with comparable accuracy.

  56. Mono-Hydra++: Real-Time Monocular Scene Graph Construction with Multi-Task Learning for 3D Indoor Mapping

    cs.RO 2026-05 unverdicted novelty 5.0

    Mono-Hydra++ is a monocular RGB-IMU pipeline that constructs hierarchical 3D scene graphs in real time while reporting lower trajectory error than some RGB-D baselines on indoor datasets.

  57. Pose-Aware Diffusion for 3D Generation

    cs.CV 2026-05 unverdicted novelty 5.0

    PAD synthesizes 3D geometry in observation space via depth unprojection as anchor to eliminate pose ambiguity in image-to-3D generation.

  58. Learning from the Unseen: Generative Data Augmentation for Geometric-Semantic Accident Anticipation

    cs.CV 2026-04 unverdicted novelty 5.0

    A generative video synthesis pipeline paired with a semantic graph neural network yields gains in accident anticipation accuracy and lead time on driving datasets, accompanied by a new benchmark release.

  59. SS3D: End2End Self-Supervised 3D from Web Videos

    cs.CV 2026-04 unverdicted novelty 5.0

    SS3D scales SfM-based self-supervision to ~100M frames from YouTube-8M using a multi-view signal proxy for filtering and a two-stage training schedule, achieving strong zero-shot transfer and better fine-tuning than p...

  60. Enhancing Hazy Wildlife Imagery: AnimalHaze3k and IncepDehazeGan

    cs.CV 2026-04 conditional novelty 5.0

    A new wildlife-specific hazy image dataset and IncepDehazeGan model that reports state-of-the-art dehazing metrics and more than doubles downstream animal detection performance.

Reference graph

Works this paper leans on

52 extracted references · 52 canonical work pages · cited by 70 Pith papers · 1 internal anchor

  1. [1]

    Stegun., ia (1972)

    Miton Abramowitz. Stegun., ia (1972). handbook of mathe- matical functions. Formulas, Graphs and Mathematical Ta- bles, 2002. 4

    work page 1972

  2. [2]

    Attention attention everywhere: Monocular depth prediction with skip attention

    Ashutosh Agarwal and Chetan Arora. Attention attention everywhere: Monocular depth prediction with skip attention. arXiv preprint arXiv:2210.09071, 2022. 3, 4

    work page arXiv 2022

  3. [3]

    BEiT: BERT Pre-Training of Image Transformers

    Hangbo Bao, Li Dong, and Furu Wei. Beit: BERT pre- training of image transformers. CoRR, abs/2106.08254,

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    Unimodal prob- ability distributions for deep ordinal classification

    Christopher Beckham and Christopher Pal. Unimodal prob- ability distributions for deep ordinal classification. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning , volume 70 of Proceedings of Machine Learning Research , pages 411–

    work page

  5. [5]

    PMLR, 06–11 Aug 2017. 4, 9

    work page 2017

  6. [6]

    Adabins: Depth estimation using adaptive bins

    Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4009–4018, 2021. 1, 2, 3, 4, 6, 7, 8, 13, 14, 15, 16, 20

    work page 2021

  7. [7]

    Localbins: Improving depth estimation by learning local dis- tributions

    Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Localbins: Improving depth estimation by learning local dis- tributions. In European Conference on Computer Vision , pages 480–496. Springer, 2022. 1, 2, 3, 4, 6, 7, 8, 9, 13, 14, 15, 16, 20

    work page 2022

  8. [8]

    Vir- tual kitti 2, 2020

    Yohann Cabon, Naila Murray, and Martin Humenberger. Vir- tual kitti 2, 2020. 5, 12, 13, 15, 17

    work page 2020

  9. [9]

    Structure- aware residual pyramid network for monocular depth esti- mation

    Xiaotian Chen, Xuejin Chen, and Zheng-Jun Zha. Structure- aware residual pyramid network for monocular depth esti- mation. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19 , pages 694–700. International Joint Conferences on Artificial Intel- ligence Organization, 7 2019. 6, 20

    work page 2019

  10. [10]

    Depth map prediction from a single image using a multi-scale deep net- work

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep net- work. In NIPS, 2014. 6, 20

    work page 2014

  11. [11]

    A review of sparse expert mod- els in deep learning,

    William Fedus, Jeff Dean, and Barret Zoph. A review of sparse expert models in deep learning. arXiv preprint arXiv:2209.01667, 2022. 5

    work page arXiv 2022

  12. [12]

    Deep ordinal regression net- work for monocular depth estimation

    Huan Fu, Mingming Gong, Chaohui Wang, Nematollah Bat- manghelich, and Dacheng Tao. Deep ordinal regression net- work for monocular depth estimation. 2018 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 2002–2011, 2018. 3, 6, 7, 20

    work page 2018

  13. [13]

    3d packing for self-supervised monocular depth estimation

    Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Allan Raven- tos, and Adrien Gaidon. 3d packing for self-supervised monocular depth estimation. In IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), 2020. 5, 12, 13, 15, 18

    work page 2020

  14. [14]

    Detail pre- serving depth estimation from a single image using attention guided networks

    Zhixiang Hao, Yu Li, Shaodi You, and Feng Lu. Detail pre- serving depth estimation from a single image using attention guided networks. 2018 International Conference on 3D Vi- sion (3DV), pages 304–313, 2018. 6, 20

    work page 2018

  15. [15]

    Revisiting single image depth estimation: Toward higher resolution maps with accurate object boundaries

    Junjie Hu, Mete Ozay, Yan Zhang, and Takayuki Okatani. Revisiting single image depth estimation: Toward higher resolution maps with accurate object boundaries. 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1043–1051, 2018. 6, 20

    work page 2019

  16. [16]

    The apolloscape open dataset for autonomous driving and its application

    Xinyu Huang, Peng Wang, Xinjing Cheng, Dingfu Zhou, Qichuan Geng, and Ruigang Yang. The apolloscape open dataset for autonomous driving and its application. IEEE transactions on pattern analysis and machine intelligence , 42(10):2702–2719, 2019. 5, 13

    work page 2019

  17. [17]

    Depth map decomposition for monocular depth estimation

    Jinyoung Jun, Jae-Han Lee, Chul Lee, and Chang-Su Kim. Depth map decomposition for monocular depth estimation. arXiv preprint arXiv:2208.10762, 2022. 2, 3, 6

    work page arXiv 2022

  18. [18]

    Deep monocular depth estimation via in- tegration of global and local predictions

    Youngjung Kim, Hyungjoo Jung, Dongbo Min, and Kwanghoon Sohn. Deep monocular depth estimation via in- tegration of global and local predictions. IEEE transactions on Image Processing, 27(8):4131–4144, 2018. 5, 12, 13, 16, 18

    work page 2018

  19. [19]

    Evaluation of cnn-based single-image depth estimation methods

    Tobias Koch, Lukas Liebel, Friedrich Fraundorfer, and Marco K¨orner. Evaluation of cnn-based single-image depth estimation methods. In Proceedings ECCV 2018 Workshops,

    work page 2018

  20. [20]

    Deeper depth prediction with fully convolutional residual networks

    Iro Laina, Christian Rupprecht, Vasileios Belagiannis, Fed- erico Tombari, and Nassir Navab. Deeper depth prediction with fully convolutional residual networks. 2016 Fourth In- ternational Conference on 3D Vision (3DV), pages 239–248,

    work page 2016

  21. [21]

    arXiv preprint arXiv:1907.10326 , year=

    Jin Han Lee, Myung-Kyu Han, Dong Wook Ko, and Il Hong Suh. From big to small: Multi-scale local planar guidance for monocular depth estimation. arXiv preprint arXiv:1907.10326, 2019. 6, 8, 13, 14, 15, 16, 20

    work page arXiv 1907

  22. [22]

    Monocular depth es- timation using relative depth maps

    Jae-Han Lee and Chang-Su Kim. Monocular depth es- timation using relative depth maps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729–9738, 2019. 2

    work page 2019

  23. [23]

    Depth- assisted real-time 3d object detection for augmented reality

    Wonwoo Lee, Nohyoung Park, and Woontack Woo. Depth- assisted real-time 3d object detection for augmented reality. ICAT’11, 2:126–132, 2011. 6, 20

    work page 2011

  24. [24]

    Monocular depth es- timation with hierarchical fusion of dilated cnns and soft- weighted-sum inference

    Bo Li, Yuchao Dai, and Mingyi He. Monocular depth es- timation with hierarchical fusion of dilated cnns and soft- weighted-sum inference. Pattern Recognition, 83:328–339,

    work page

  25. [25]

    Deep attention-based classification network for robust depth prediction

    Ruibo Li, Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, and Lingxiao Hang. Deep attention-based classification network for robust depth prediction. In C.V . Jawahar, Hongdong Li, Greg Mori, and Konrad Schindler, editors, Computer Vision – ACCV 2018, pages 663–678, Cham, 2019. Springer Inter- national Publishing. 3

    work page 2018

  26. [26]

    Megadepth: Learning single- view depth prediction from internet photos

    Zhengqi Li and Noah Snavely. Megadepth: Learning single- view depth prediction from internet photos. In Computer Vision and Pattern Recognition (CVPR), 2018. 5, 13

    work page 2018

  27. [27]

    Binsformer: Revisiting adaptive bins for monocular depth estimation,

    Zhenyu Li, Xuyang Wang, Xianming Liu, and Junjun Jiang. Binsformer: Revisiting adaptive bins for monocular depth estimation. arXiv preprint arXiv:2204.00987, 2022. 1, 2, 3, 4

    work page arXiv 2022

  28. [28]

    Swin transformer v2: Scaling up capacity and resolution

    Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 12009–12019, 2022. 8, 12 10

    work page 2022

  29. [29]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012–10022, 2021. 8

    work page 2021

  30. [30]

    Object scene flow for au- tonomous vehicles

    Moritz Menze and Andreas Geiger. Object scene flow for au- tonomous vehicles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June

    work page

  31. [31]

    Single image depth estimation: An overview

    Alican Mertan, Damien Jade Duff, and Gozde Unal. Single image depth estimation: An overview. Digital Signal Pro- cessing, 123:103441, 2022. 1, 2

    work page 2022

  32. [32]

    Sharpnet: Fast and accurate recovery of occluding contours in monocular depth estimation

    Michael Ramamonjisoa and Vincent Lepetit. Sharpnet: Fast and accurate recovery of occluding contours in monocular depth estimation. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision (ICCV) Workshops , Oct 2019. 6, 20

    work page 2019

  33. [33]

    Vi- sion transformers for dense prediction

    Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 12179–12188, October 2021. 2, 3, 5

    work page 2021

  34. [34]

    Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer

    Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence (TPAMI), 2020. 1, 2, 3, 4, 5, 12, 13

    work page 2020

  35. [35]

    Deep robust single image depth estimation neural network using scene understanding

    Haoyu Ren, Mostafa El-Khamy, and Jungwon Lee. Deep robust single image depth estimation neural network using scene understanding. In CVPR Workshops, 2019. 3

    work page 2019

  36. [36]

    Susskind

    Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M. Susskind. Hypersim: A photorealistic syn- thetic dataset for holistic indoor scene understanding. In International Conference on Computer Vision (ICCV) 2021,

    work page 2021

  37. [37]

    Progress and proposals: A case study of monocular depth estimation

    Khalil Sarwari, Forrest Laine, and Claire Tomlin. Progress and proposals: A case study of monocular depth estimation. Master’s thesis, EECS Department, University of California, Berkeley, May 2021. 3, 7

    work page 2021

  38. [38]

    Indoor segmentation and support inference from rgbd images

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In Computer Vision – ECCV 2012, pages 746– 760, Berlin, Heidelberg, 2012. Springer Berlin Heidelberg. 2, 5, 6, 12, 13

    work page 2012

  39. [39]

    S. Song, S. P. Lichtenberg, and J. Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In 2015 IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR) , pages 567–576, 2015. 5, 12, 13, 14

    work page 2015

  40. [40]

    Mingxing Tan and Quoc V . Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In Ka- malika Chaudhuri and Ruslan Salakhutdinov, editors, Pro- ceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, Cali- fornia, USA, volume 97 ofProceedings of Machine Learning Research, pages 6105–...

    work page 2019

  41. [41]

    Z., Daniele, A

    Igor Vasiljevic, Nick Kolkin, Shanyi Zhang, Ruotian Luo, Haochen Wang, Falcon Z. Dai, Andrea F. Daniele, Moham- madreza Mostajabi, Steven Basart, Matthew R. Walter, and Gregory Shakhnarovich. DIODE: A Dense Indoor and Out- door DEpth Dataset. CoRR, abs/1908.00463, 2019. 5, 12, 13, 14, 16, 19

    work page arXiv 1908

  42. [42]

    Web stereo video supervision for depth prediction from dynamic scenes

    Chaoyang Wang, Simon Lucey, Federico Perazzi, and Oliver Wang. Web stereo video supervision for depth prediction from dynamic scenes. In 2019 International Conference on 3D Vision (3DV), pages 348–357. IEEE, 2019. 5, 13

    work page 2019

  43. [43]

    Irs: A large naturalistic indoor robotics stereo dataset to train deep models for dis- parity and surface normal estimation,

    Qiang Wang, Shizhen Zheng, Qingsong Yan, Fei Deng, Kaiyong Zhao, and Xiaowen Chu. Irs: A large naturalis- tic indoor robotics stereo dataset to train deep models for disparity and surface normal estimation. arXiv preprint arXiv:1912.09678, 2019. 5, 13

    work page arXiv 1912

  44. [44]

    Tartanair: A dataset to push the limits of visual slam

    Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Se- bastian Scherer. Tartanair: A dataset to push the limits of visual slam. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4909–4916. IEEE, 2020. 5, 13

    work page 2020

  45. [45]

    Pytorch image models

    Ross Wightman. Pytorch image models. https : / / github . com / rwightman / pytorch - image - models, 2019. 5, 12

    work page 2019

  46. [46]

    Monocular relative depth percep- tion with web stereo data supervision

    Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, Yang Xiao, Ruibo Li, and Zhenbo Luo. Monocular relative depth percep- tion with web stereo data supervision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 311–320, 2018. 5, 13

    work page 2018

  47. [47]

    Structure-guided ranking loss for single im- age depth prediction

    Ke Xian, Jianming Zhang, Oliver Wang, Long Mai, Zhe Lin, and Zhiguo Cao. Structure-guided ranking loss for single im- age depth prediction. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 611–620, 2020. 5, 13

    work page 2020

  48. [48]

    Blendedmvs: A large- scale dataset for generalized multi-view stereo networks

    Yao Yao, Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, and Long Quan. Blendedmvs: A large- scale dataset for generalized multi-view stereo networks. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition , pages 1790–1799, 2020. 5, 13

    work page 2020

  49. [49]

    En- forcing geometric constraints of virtual normal for depth pre- diction

    Wei Yin, Yifan Liu, Chunhua Shen, and Youliang Yan. En- forcing geometric constraints of virtual normal for depth pre- diction. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV), October 2019. 6, 20

    work page 2019

  50. [50]

    Learning to recover 3d scene shape from a single image

    Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Long Mai, Simon Chen, and Chunhua Shen. Learning to recover 3d scene shape from a single image. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 204–213, 2021. 3

    work page 2021

  51. [51]

    New crfs: Neural window fully-connected crfs for monocular depth estimation,

    Weihao Yuan, Xiaodong Gu, Zuozhuo Dai, Siyu Zhu, and Ping Tan. New crfs: Neural window fully-connected crfs for monocular depth estimation. arXiv preprint arXiv:2203.01502, 2022. 1, 2, 6, 7, 8, 13, 14, 15, 16, 17, 18, 19, 20 11 A. Appendix A.1. Datasets Overview We begin by providing a detailed overview of the prop- erties of the datasets used for metric ...

    work page arXiv 2022

  52. [52]

    When using the base and tiny variants Swin2-B and Swin2-T, the number of parame- ters of ZoeDepth drops to 102M and 42M, respectively

    based on shifted windows. When using the base and tiny variants Swin2-B and Swin2-T, the number of parame- ters of ZoeDepth drops to 102M and 42M, respectively. We report the results of all the aforementioned models evalu- ated on NYU Depth V2 in Table 18. 12 Seen in # Train # Eval Eval Depth [m] Crop Dataset Domain Type Training? Samples Samples Min Max ...

    work page