pith. sign in

arxiv: 2605.28048 · v1 · pith:BIJ36J6Vnew · submitted 2026-05-27 · 💻 cs.RO

SAFEVPR: Patch-Based Conformal Verification for Safe Cross-Condition Sequence Visual Place Recognition

Ha Sier , Jiaqiang Zhang , Zhuo Zou , Xianjia Yu , Tomi Westerlund This is my paper

Pith reviewed 2026-06-29 12:15 UTC · model grok-4.3

classification 💻 cs.RO
keywords visual place recognitionconformal predictionfalse discovery ratecross-condition shiftspatch matchingsequence VPRMondrian calibrationrobot relocalization
0
0 comments X

The pith

A patch-based mutual nearest neighbour score combined with Mondrian conformal calibration maintains empirical false discovery rate control for sequence visual place recognition under cross-condition shifts.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces SAFEVPR as a non-trainable pipeline that decides whether to accept the top retrieved place match in robotics tasks such as SLAM and relocalization. It replaces standard cosine similarity with a mutual nearest neighbour patch matching score drawn from frozen DINOv2 features and applies separate Bonferroni-corrected thresholds to different score bins via Mondrian conformal Learn-Then-Test calibration. This construction is intended to deliver finite-sample false discovery rate guarantees when calibration and test data are exchangeable, and the authors test whether the same thresholds remain valid under the condition shifts typical of real deployments. The work matters because unreliable match acceptance can cause navigation errors, and the method provides a concrete way to bound those errors without retraining models. Evaluation on 23 cross-condition pairs from three datasets shows the target error rate is met in every case while still accepting a useful fraction of matches.

Core claim

SAFEVPR replaces the backbone cosine similarity with a mutual-nearest-neighbour patch-matching score computed from frozen DINOv2 ViT features and replaces flat Learn-Then-Test calibration with Mondrian conformal LTT that fits separate Bonferroni-corrected thresholds across score bins. Under exchangeability these thresholds would provide finite-sample false-discovery-rate control; under condition shift the paper evaluates empirical validity per deployment. Across 23 cross-condition setups from Oxford RobotCar, NCLT, and St Lucia datasets, using three frozen VPR backbones, the method is empirically valid on all 23 setups at target FDR alpha = 0.10, achieving mean accepted FDR 0.014 and mean tr

What carries the argument

Mutual-nearest-neighbour patch-matching score from frozen DINOv2 features, paired with Mondrian conformal Learn-Then-Test calibration that fits separate Bonferroni-corrected thresholds per score bin.

If this is right

  • Sequence VPR systems can include an accept/reject step that empirically bounds the false discovery rate even when lighting, weather, or seasonal conditions differ from calibration.
  • High overall discrimination measured by AUROC does not guarantee conformal validity, as shown by the failure of certain high-AUROC backbones under the same calibration.
  • On textureless or repetitive scenery the pipeline abstains rather than accepting unreliable matches.
  • The non-trainable design allows the verification layer to be added to any existing frozen VPR backbone without retraining.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same per-bin conformal calibration strategy could be tested on other perception modules that must decide acceptance under distribution shift, such as loop-closure detection in mapping.
  • Success on the chosen datasets indicates that representative calibration data is essential; a practical next step would be to examine how much new data is required when a previously unseen shift appears.
  • If future deployments encounter condition changes more extreme than those in the 23 pairs, the current thresholds would require fresh calibration on data reflecting the new shift.

Load-bearing premise

The assumption that score binning and Bonferroni-corrected Mondrian thresholds fitted on the calibration splits of these three datasets will continue to deliver the reported FDR control for arbitrary unseen condition shifts.

What would settle it

A new cross-condition deployment, using the same calibration procedure, in which the realized false discovery rate among accepted matches exceeds the target alpha of 0.10.

Figures

Figures reproduced from arXiv: 2605.28048 by Ha Sier, Jiaqiang Zhang, Tomi Westerlund, Xianjia Yu, Zhuo Zou.

Figure 1
Figure 1. Figure 1: The SAFEVPR pipeline. Stage 1 (grey): a frozen VPR backbone and diagonal-DTW (dynamic time warping) matcher return a top-1 candidate c ⋆ , treated as a black box (dashed); the cosine score scos is discarded. Stage 2 (orange, ours): a non-trainable DINOv2 patch verifier matches query and candidate patches by mutual nearest neighbour (MNN) with a Lowe-ratio test, yielding a bounded score s ∈ [0, 1]. Stage 3 … view at source ↗
Figure 2
Figure 2. Figure 2: Cal-vs-test score distributions for the four verifiers in Ta [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Per-configuration bootstrap mean FDR with [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
read the original abstract

Sequence-based visual place recognition (VPR) for SLAM and robot relocalization must decide whether the retrieved top-1 candidate is safe to accept. Conformal prediction is a natural framework for this accept/reject decision, but its finite-sample guarantees rely on exchangeability between calibration and deployment (test) data, which is violated under cross-condition deployment. We introduce SAFEVPR, a non-trainable verification-and-calibration pipeline for safe cross-condition sequence VPR. SAFEVPR replaces the standard backbone cosine similarity with a mutual-nearest-neighbour (MNN) patch-matching score computed from frozen DINOv2 ViT features, and replaces flat Learn-Then-Test calibration with Mondrian conformal LTT, fitting separate Bonferroni-corrected thresholds across score bins. Under exchangeability, these thresholds would provide finite-sample false-discovery-rate (FDR) control; under condition shift, we evaluate empirical validity per deployment. Across 23 cross-condition setups from Oxford RobotCar, NCLT, and St Lucia datasets, using three frozen VPR backbones, SAFEVPR is empirically valid on 23/23 setups at target FDR alpha = 0.10, achieving mean accepted FDR 0.014 and mean true-positive rate (TPR) 0.75. The results show that raw discrimination alone is not sufficient for conformal validity: AnyLoc-VLAD and Super-Point+LightGlue reach comparable area under the receiver operating characteristic curve (AUROC) but fail more setups under the same calibration. On textureless repetitive scenery, SAFEVPR safely abstains rather than accepting unreliable matches. Code is available at https://github.com/Hasar12139/SafeVPR.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces SAFEVPR, a non-trainable verification pipeline for sequence VPR that substitutes cosine similarity with mutual-nearest-neighbour patch-matching scores derived from frozen DINOv2 ViT features and employs Mondrian conformal Learn-Then-Test calibration with per-bin Bonferroni-corrected thresholds. The central empirical claim is that this yields valid FDR control at target alpha=0.10 on all 23 cross-condition test deployments drawn from Oxford RobotCar, NCLT and St Lucia (using three backbones), with mean accepted FDR 0.014 and mean TPR 0.75; raw AUROC is shown to be insufficient for conformal validity.

Significance. If the reported per-deployment empirical validity holds, the work supplies a practical, calibration-only method for safe accept/reject decisions in cross-condition VPR that is directly relevant to SLAM and robot relocalization. The explicit contrast between discrimination metrics and conformal validity, together with public code, strengthens the contribution for robotics applications.

major comments (2)
  1. [Evaluation] Evaluation section: empirical FDR control is demonstrated exclusively on the 23 condition shifts internal to the three calibration datasets; the fixed bin boundaries and Mondrian thresholds therefore carry no finite-sample or distribution-free guarantee for arbitrary unseen shifts (different sensors, lighting regimes or environments) outside this span, which is the load-bearing assumption for the 'safe cross-condition' claim.
  2. [Method] Method section on Mondrian LTT: the precise construction of score bins, any score normalization step, and the exact procedure for computing 'accepted FDR' from the test sequences are not stated with sufficient formality to allow independent reproduction or verification of how the Bonferroni correction is applied across bins.
minor comments (2)
  1. [Abstract] Abstract and §1: the term 'accepted FDR' is used before its operational definition is given; a brief parenthetical or forward reference would improve readability.
  2. [Figures/Tables] Figure captions and tables: axis labels and legend entries for the per-setup FDR/TPR plots could be expanded to include the exact alpha value and backbone names for standalone clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. Below we address each major point directly, with planned revisions where appropriate.

read point-by-point responses

  1. Referee: [Evaluation] Evaluation section: empirical FDR control is demonstrated exclusively on the 23 condition shifts internal to the three calibration datasets; the fixed bin boundaries and Mondrian thresholds therefore carry no finite-sample or distribution-free guarantee for arbitrary unseen shifts (different sensors, lighting regimes or environments) outside this span, which is the load-bearing assumption for the 'safe cross-condition' claim.

    Authors: We agree that no distribution-free guarantee exists for shifts outside the evaluated span, as conformal validity fundamentally requires exchangeability. The manuscript's claim is strictly empirical: valid FDR control on all 23 cross-condition setups from the three datasets. We will revise the abstract, introduction, and conclusion to state explicitly that 'safe cross-condition' refers to demonstrated empirical performance across these diverse internal condition shifts, without claiming guarantees for arbitrary unseen sensors or environments. revision: yes

  2. Referee: [Method] Method section on Mondrian LTT: the precise construction of score bins, any score normalization step, and the exact procedure for computing 'accepted FDR' from the test sequences are not stated with sufficient formality to allow independent reproduction or verification of how the Bonferroni correction is applied across bins.

    Authors: We acknowledge the need for greater formality. In revision we will add: score bins are formed by quantile partitioning of calibration MNN scores into a fixed number of bins; MNN scores undergo min-max normalization to [0,1]; accepted FDR is the empirical proportion of false positives among accepted test matches; Bonferroni correction divides target alpha by the number of bins to obtain per-bin thresholds. A formal algorithm box with equations will be included. revision: yes

Circularity Check

0 steps flagged

No significant circularity; result is empirical evaluation on public datasets using standard conformal procedures.

full rationale

The paper presents SAFEVPR as a non-trainable pipeline that applies Mondrian conformal LTT with Bonferroni correction to score bins derived from calibration splits. The central claim is empirical validity (FDR control observed on 23/23 cross-condition test pairs drawn from the same three datasets). No derivation chain reduces by the paper's equations to a fitted parameter renamed as prediction, nor does any load-bearing step rely on self-citation of an unverified uniqueness result. Calibration thresholds are produced by the standard procedure on held-out data and then evaluated on separate deployments; the reported numbers are direct measurements, not forced by construction. The paper explicitly distinguishes the exchangeability case (finite-sample guarantee) from the condition-shift case (empirical check), avoiding any claim that the thresholds are theoretically guaranteed outside the evaluated shifts.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The approach relies on the standard theoretical guarantee of Mondrian conformal prediction within score bins; no new free parameters, invented physical entities, or ad-hoc axioms beyond that domain assumption are introduced.

axioms (1)
  • domain assumption Mondrian conformal prediction with Bonferroni correction controls FDR when exchangeability holds within each score bin
    Invoked to justify replacing flat LTT with per-bin thresholds under the condition-shift regime.

pith-pipeline@v0.9.1-grok · 5853 in / 1493 out tokens · 46920 ms · 2026-06-29T12:15:19.747341+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    V ovk, A

    V . V ovk, A. Gammerman, and G. Shafer,Algorithmic Learning in a Random World. Springer, 2005

    2005

  2. [2]

    Learn then test: Calibrating predictive algorithms to achieve risk control,

    A. N. Angelopoulos, S. Bates, E. J. Candès, M. I. Jordan, and L. Lei, “Learn then test: Calibrating predictive algorithms to achieve risk control,” 2021

    2021

  3. [3]

    DINOv2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandezet al., “DINOv2: Learning robust visual features without supervision,”Transactions on Machine Learning Research (TMLR), 2024

    2024

  4. [4]

    Mondrian confidence machine,

    V . V ovk, D. Lindsay, I. Nouretdinov, and A. Gammerman, “Mondrian confidence machine,” inTech. Report, 2003

    2003

  5. [5]

    1 year, 1000 km: The Oxford RobotCar dataset,

    W. Maddern, G. Pascoe, C. Linegar, and P. Newman, “1 year, 1000 km: The Oxford RobotCar dataset,”The International Journal of Robotics Research, vol. 36, no. 1, pp. 3–15, 2017

    2017

  6. [6]

    University of Michigan North Campus long-term vision and lidar dataset,

    N. Carlevaris-Bianco, A. K. Ushani, and R. M. Eustice, “University of Michigan North Campus long-term vision and lidar dataset,”The International Journal of Robotics Research, vol. 35, no. 9, pp. 1023– 1035, 2016

    2016

  7. [7]

    FAB- MAP+RatSLAM: Appearance-based SLAM for multiple times of day,

    A. J. Glover, W. P. Maddern, M. J. Milford, and G. F. Wyeth, “FAB- MAP+RatSLAM: Appearance-based SLAM for multiple times of day,” inIEEE International Conference on Robotics and Automation (ICRA), 2010

    2010

  8. [8]

    Deep visual geo-localization benchmark,

    G. Berton, R. Mereu, G. Trivigno, C. Masone, G. Csurka, T. Sattler, and B. Caputo, “Deep visual geo-localization benchmark,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  9. [9]

    NetVLAD: CNN architecture for weakly supervised place recognition,

    R. Arandjelovi ´c, P. Gronat, A. Torii, T. Pajdla, and J. Sivic, “NetVLAD: CNN architecture for weakly supervised place recognition,”IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 40, no. 6, pp. 1437– 1451, 2018

    2018

  10. [10]

    Rethinking visual geo- localization for large-scale applications,

    G. Berton, C. Masone, and B. Caputo, “Rethinking visual geo- localization for large-scale applications,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  11. [11]

    D 2-VPR: Dual decoupling network for visual place recog- nition,

    G. Luet al., “D 2-VPR: Dual decoupling network for visual place recog- nition,” inIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024

    2024

  12. [12]

    AnyLoc: Towards universal visual place recognition,

    N. Keetha, A. Mishra, J. Karhade, K. M. Jatavallabhula, S. Scherer, M. Krishna, and S. Garg, “AnyLoc: Towards universal visual place recognition,” inIEEE Robotics and Automation Letters (RA-L), 2024

    2024

  13. [13]

    EffoVPR: Effective foundation model utilization for visual place recognition,

    S. Tziony, A. Cohen, D. Aiger, and P. Dollár, “EffoVPR: Effective foundation model utilization for visual place recognition,” 2024

    2024

  14. [14]

    Towards seamless adaptation of pre-trained models for visual place recognition,

    F. Lu, L. Zhang, X. Lan, S. Dong, Y . Wang, and C. Yuan, “Towards seamless adaptation of pre-trained models for visual place recognition,” inInternational Conference on Learning Representations (ICLR), 2024

    2024

  15. [15]

    Patch- NetVLAD: Multi-scale fusion of locally-global descriptors for place recognition,

    S. Hausler, S. Garg, M. Xu, M. Milford, and T. Fischer, “Patch- NetVLAD: Multi-scale fusion of locally-global descriptors for place recognition,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 14 141–14 152

    2021

  16. [16]

    A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification

    A. N. Angelopoulos and S. Bates, “A gentle introduction to confor- mal prediction and distribution-free uncertainty quantification,”arXiv preprint arXiv:2107.07511, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  17. [17]

    Conformal risk control,

    A. N. Angelopoulos, S. Bates, A. Fisch, L. Lei, and T. Schuster, “Conformal risk control,” inInternational Conference on Learning Representations (ICLR), 2024

    2024

  18. [18]

    Mondrian conformal regressors,

    H. Boström, H. Linusson, T. Löfström, and U. Johansson, “Mondrian conformal regressors,”Proceedings of Machine Learning Research, 2018

    2018

  19. [19]

    Seeing with partial certainty: Conformal prediction for robotic scene recognition in built environments,

    S. Tellexet al., “Seeing with partial certainty: Conformal prediction for robotic scene recognition in built environments,” 2025

    2025

  20. [20]

    Conformal prediction under covariate shift,

    R. J. Tibshirani, R. F. Barber, E. J. Candès, and A. Ramdas, “Conformal prediction under covariate shift,” inAdvances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  21. [21]

    Distinctive image features from scale-invariant keypoints,

    D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004

    2004

  22. [22]

    Large-scale location recognition and the geometric burstiness problem,

    T. Sattler, M. Havlena, K. Schindler, and M. Pollefeys, “Large-scale location recognition and the geometric burstiness problem,”IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017

    2017

  23. [23]

    LightGlue: Local feature matching at light speed,

    P. Lindenberger, P.-E. Sarlin, and M. Pollefeys, “LightGlue: Local feature matching at light speed,” inIEEE/CVF International Conference on Computer Vision (ICCV), 2023

    2023