pith. sign in

arxiv: 2604.15652 · v2 · pith:2ZJTLHESnew · submitted 2026-04-17 · 💻 cs.CV

Towards Realistic Open-Vocabulary Remote Sensing Segmentation: Benchmark and Baseline

Bingyu Li , Tao Huo , Haocheng Dong , Da Zhang , Zhiyuan Zhao , Junyu Gao , Xuelong Li This is my paper

Pith reviewed 2026-05-10 08:16 UTC · model grok-4.3

classification 💻 cs.CV
keywords open-vocabulary segmentationremote sensingbenchmark datasetpositive-incentive noiseimage segmentationOVRSISBenchV2transfer learning
0
0 comments X

The pith

OVRSISBenchV2 provides a more realistic and challenging benchmark for open-vocabulary remote sensing segmentation, where the Pi-Seg baseline achieves strong performance through positive-incentive noise.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper seeks to overcome the fragmented and limited datasets in open-vocabulary remote sensing image segmentation by creating OVRSISBenchV2. This new benchmark combines a new 95K image dataset with 10 others to reach 170K images and 128 categories, plus protocols for practical tasks like building extraction. A sympathetic reader would care because current evaluations may not reflect real geospatial needs, leading to methods that fail in deployment. The authors also present Pi-Seg, which uses a positive-incentive noise mechanism to improve how visual and text features align during training. If this holds, it suggests that both better data and targeted perturbations can push forward generalization in this domain.

Core claim

OVRSISBenchV2 substantially expands scene diversity, semantic coverage, and evaluation difficulty beyond prior work, and Pi-Seg delivers strong and consistent results particularly on the more challenging OVRSISBenchV2 benchmark by using positive-incentive noise to broaden the visual-text feature space.

What carries the argument

The positive-incentive noise mechanism in Pi-Seg, consisting of learnable and semantically guided perturbations that broaden the visual-text feature space during training to improve transferability.

If this is right

  • OVRSISBenchV2 enables more rigorous testing of open-vocabulary methods in remote sensing.
  • Pi-Seg shows consistent improvements across OVRSISBenchV1, V2, and downstream tasks.
  • Realistic benchmarks better reflect complex deployment scenarios like flood detection.
  • Positive-incentive noise enhances cross-dataset generalization in segmentation models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar benchmark expansion approaches could be applied to other specialized domains like medical imaging.
  • The noise perturbation idea might transfer to general open-vocabulary segmentation outside remote sensing.
  • Future work could test if the benchmark's diversity reduces selection bias in model development.
  • Integration with other sensors or modalities could further test the method's robustness.

Load-bearing premise

The manually selected categories and datasets accurately capture the full range of realistic geospatial application demands without introducing selection bias or missing key scene and sensor variations.

What would settle it

A follow-up experiment where Pi-Seg fails to outperform standard baselines on OVRSISBenchV2 or where new datasets reveal that the 128 categories do not cover important real-world cases would challenge the central claims.

Figures

Figures reproduced from arXiv: 2604.15652 by Bingyu Li, Da Zhang, Haocheng Dong, Junyu Gao, Tao Huo, Xuelong Li, Zhiyuan Zhao.

Figure 1
Figure 1. Figure 1: From OVRSISBenchV1/RSKT-Seg (V1 Paper) to OVRSISBenchV2/Pi-Seg(V2 Paper). (a.1) OVRSISBenchV1 adopts single-source cross-dataset evaluation. (a.2) RSKT-Seg improves OVRSIS through remote-sensing-aware cost-map construction and refinement. (b) OVRSISBenchV2 expands the training foundation to 95K pairs across five scene domains, while Pi-Seg provides a noise-aware baseline with stronger generalization under … view at source ↗
Figure 2
Figure 2. Figure 2: Two-stage pipeline for OVRSIS95K Construction. Given an input remote sensing image, Stage 1 first generates a detailed scene caption and parses it into candidate object categories, which are then aligned with the dataset taxonomy through filtering and matching. Stage 2 uses the resulting categories to produce instance-level masks, followed by mask combination and correction to obtain the final semantic-lev… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of OVRSIS95K. The dataset is organized into five representative remote sensing scene domains: wasteland, industrial, town, waterfront, and forest. The upper part provides semantic descriptions and characteristic examples for each domain, while the lower part presents representative image patches with corresponding dense annotations. open-vocabulary segmentation into specialized domains such as rem… view at source ↗
Figure 4
Figure 4. Figure 4: Evolutionary overview of the OVRSISBench series. OVRSISBenchV1 (left) focuses on standard open-vocabulary protocols with a primary emphasis on basic remote sensing datasets. In contrast, OVRSISBenchV2 (right) represents a significant expansion in both scale and diversity, utilizing the comprehensive OVRSIS95K dataset for training and incorporating multiple specialized downstream tasks to evaluate model gen… view at source ↗
Figure 6
Figure 6. Figure 6: Comparison between RSKT-Seg and Pi-Seg. (a) RSKT [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Overview of the proposed Pi-Seg. (a) Main pipeline. CLIP image and text encoders extract visual and textual features, which is perturbed by two semantic-aware perturbation module (SPM): Image-SPM and Text-SPM, followed by cost-map construction and upsampling-based decoding. (b) Text perturbation. A learnable noise generator transforms deterministic text embeddings into stochastic ones to improve semantic r… view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative comparison of segmentation results on OVRSISBenchV1 and OVRSISBenchV2. The left panel shows representative results on OVRSISBenchV1, while the right panel presents corresponding examples on OVRSISBenchV2. We compare the proposed Pi-Seg with several strong baselines. Across both benchmarks, Pi-Seg produces segmentation maps that are more spatially complete, semantically coherent, and better alig… view at source ↗
Figure 9
Figure 9. Figure 9: Visualization of Downstream Tasks: Building Extraction, Flood Extraction, Road Extraction. performance on both high-resolution m-mIoU/m-mACC and low-resolution m-mIoU, while remaining highly competitive on low-resolution m-mACC. This suggests that Pi-Seg generalizes well across different image scales and is particularly effective for high-resolution remote sensing imagery, where fine-grained structures and… view at source ↗
Figure 10
Figure 10. Figure 10: Training dynamics of the proposed Pi-Seg measured by four Δ-correlation statistics. We report gt_in_mean, non_gt_mean, gap, and align_ratio throughout training. From left to right, the curves correspond to training on DLRSD, iSAID, and OVRSIS95K. As training progresses, the learned perturbation strengthens correlation responses in ground-truth regions while suppressing those in non-target regions, resulti… view at source ↗
Figure 11
Figure 11. Figure 11: Qualitative examples of fine-grained cost refinement with the proposed Pi-Seg. Compared with the original cost maps, Pi-Noise introduces class-aware corrections that highlight target regions and suppress spurious background responses. The refined cost maps are more spatially concentrated and better aligned with binary ground truth. background suppression, effectively reducing false activation over visuall… view at source ↗
Figure 12
Figure 12. Figure 12: Representative failure cases of Pi-Seg. Pi-Seg may [PITH_FULL_IMAGE:figures/full_fig_p015_12.png] view at source ↗
read the original abstract

Open-vocabulary remote sensing image segmentation (OVRSIS) remains underexplored due to fragmented datasets, limited training diversity, and the lack of evaluation benchmarks that reflect realistic geospatial application demands. Our previous \textit{OVRSISBenchV1} established an initial cross-dataset evaluation protocol, but its limited scope is insufficient for assessing realistic open-world generalization. To address this issue, we propose \textit{OVRSISBenchV2}, a large-scale and application-oriented benchmark for OVRSIS. We first construct \textbf{OVRSIS95K}, a balanced dataset of about 95K image--mask pairs covering 35 common semantic categories across diverse remote sensing scenes. Built upon OVRSIS95K and 10 downstream datasets, OVRSISBenchV2 contains 170K images and 128 categories, substantially expanding scene diversity, semantic coverage, and evaluation difficulty. Beyond standard open-vocabulary segmentation, it further includes downstream protocols for building extraction, road extraction, and flood detection, thereby better reflecting realistic geospatial application demands and complex deployment scenarios. We also propose \textbf{Pi-Seg}, a baseline for OVRSIS. Pi-Seg improves transferability through a \textbf{positive-incentive noise} mechanism, where learnable and semantically guided perturbations broaden the visual-text feature space during training. Extensive experiments on OVRSISBenchV1, OVRSISBenchV2, and downstream tasks show that Pi-Seg delivers strong and consistent results, particularly on the more challenging OVRSISBenchV2 benchmark. Our results highlight both the importance of realistic benchmark design and the effectiveness of perturbation-based transfer for OVRSIS. The code and datasets are available at \href{https://github.com/LiBingyu01/Pi-Seg}{LiBingyu01/Pi-Seg}.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

Central claims rest on standard computer-vision assumptions about feature alignment between images and text plus the representativeness of the assembled remote-sensing data; the positive-incentive noise is a new training device without independent prior validation.

axioms (1)
  • domain assumption Visual and text encoders can be aligned via contrastive or perturbation-based training to support open-vocabulary generalization.
    Invoked in the description of Pi-Seg broadening the visual-text feature space.
invented entities (1)
  • positive-incentive noise mechanism no independent evidence
    purpose: Learnable semantically guided perturbations to broaden visual-text feature space during training.
    New device introduced in the paper to improve transferability; no external falsifiable evidence provided in abstract.

pith-pipeline@v0.9.0 · 5662 in / 1288 out tokens · 42348 ms · 2026-05-10T08:16:55.367852+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.